2023-11-28T10:55:27.992 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T10:55:27.992 DEBUG:teuthology.run:Teuthology command: teuthology --name neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953 --verbose -- /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/orig.config.yaml 2023-11-28T10:55:28.038 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T10:55:28.116 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469953' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9437 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_10:48:03 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T10:55:28.117 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa; will attempt to use it 2023-11-28T10:55:28.117 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks 2023-11-28T10:55:28.118 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T10:55:28.119 INFO:teuthology.task.internal:Checking packages... 2023-11-28T10:55:28.144 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T10:55:28.145 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T10:55:28.145 INFO:teuthology.packaging:ref: None 2023-11-28T10:55:28.145 INFO:teuthology.packaging:tag: None 2023-11-28T10:55:28.145 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T10:55:28.145 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T10:55:28.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T10:55:28.422 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T10:55:28.422 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T10:55:28.428 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T10:55:28.428 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T10:55:28.435 INFO:teuthology.task.internal:Saving configuration 2023-11-28T10:55:28.450 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T10:55:28.457 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T10:55:28.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:55.428636', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:55:28.501 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:55.427071', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:55:28.501 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T10:55:28.508 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T10:55:28.508 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T10:55:28.508 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T10:55:28.581 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ff5902a60>, signals=[15]) 2023-11-28T10:55:28.581 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T10:55:28.587 INFO:teuthology.task.internal:Opening connections... 2023-11-28T10:55:28.588 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T10:55:28.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:28.657 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T10:55:28.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:28.724 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T10:55:28.730 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2023-11-28T10:55:28.750 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2023-11-28T10:55:28.750 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2023-11-28T10:55:28.808 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:55:28.808 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:ID="rhel" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="fedora" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="8.6" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:28.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T10:55:28.810 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:28.810 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:55:28.810 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:55:28.810 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:55:28.811 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2023-11-28T10:55:28.833 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2023-11-28T10:55:28.853 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2023-11-28T10:55:28.853 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2023-11-28T10:55:28.913 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:ID="rhel" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="fedora" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8.6" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:55:28.914 INFO:teuthology.orchestra.run.smithi090.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:55:28.915 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:28.915 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T10:55:28.915 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:28.915 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:55:28.915 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:55:28.915 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:55:28.915 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2023-11-28T10:55:28.933 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T10:55:28.943 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T10:55:28.948 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T10:55:28.949 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:28.951 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:28.972 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T10:55:28.978 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T10:55:28.978 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:29.011 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:29.042 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T10:55:29.090 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T10:55:29.105 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T10:55:29.105 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-28T10:55:29.105 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-28T10:55:29.106 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:29.106 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:29.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:29.125 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2023-11-28T10:55:29.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:29.126 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2023-11-28T10:55:29.184 INFO:teuthology.orchestra.run.smithi090.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:29.184 INFO:teuthology.task.kernel:Running kernel on smithi090: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:29.184 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2023-11-28T10:55:29.186 INFO:teuthology.orchestra.run.smithi083.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:29.186 INFO:teuthology.task.kernel:Running kernel on smithi083: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:29.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2023-11-28T10:55:30.333 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:30.333 INFO:teuthology.orchestra.run.smithi083.stdout:Unable to read consumer identity 2023-11-28T10:55:30.501 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:30.501 INFO:teuthology.orchestra.run.smithi090.stdout:Unable to read consumer identity 2023-11-28T10:55:31.769 INFO:teuthology.orchestra.run.smithi083.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-11-28T10:55:32.445 INFO:teuthology.orchestra.run.smithi090.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-11-28T10:55:32.495 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-28T10:55:33.237 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-11-28T10:55:35.962 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 351 kB/s | 24 kB 00:00 2023-11-28T10:55:36.624 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T10:55:36.681 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-11-28T10:55:36.683 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-11-28T10:55:36.683 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-11-28T10:55:36.771 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 434 kB/s | 24 kB 00:00 2023-11-28T10:55:36.812 DEBUG:teuthology.orchestra.run.smithi083:> echo no | sudo yum reinstall kernel || true 2023-11-28T10:55:37.141 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:37.142 INFO:teuthology.orchestra.run.smithi083.stdout:Unable to read consumer identity 2023-11-28T10:55:37.401 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:35 AM UTC. 2023-11-28T10:55:37.441 INFO:teuthology.orchestra.run.smithi083.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:37.474 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T10:55:37.475 INFO:teuthology.orchestra.run.smithi083.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:37.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum reinstall -y kernel || true 2023-11-28T10:55:37.523 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-11-28T10:55:37.523 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-11-28T10:55:37.523 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-11-28T10:55:37.646 DEBUG:teuthology.orchestra.run.smithi090:> echo no | sudo yum reinstall kernel || true 2023-11-28T10:55:37.850 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:37.850 INFO:teuthology.orchestra.run.smithi083.stdout:Unable to read consumer identity 2023-11-28T10:55:37.978 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:37.978 INFO:teuthology.orchestra.run.smithi090.stdout:Unable to read consumer identity 2023-11-28T10:55:38.103 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:03 ago on Tue 28 Nov 2023 10:55:35 AM UTC. 2023-11-28T10:55:38.142 INFO:teuthology.orchestra.run.smithi083.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:38.172 INFO:teuthology.orchestra.run.smithi083.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:38.223 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T10:55:38.241 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:36 AM UTC. 2023-11-28T10:55:38.288 INFO:teuthology.orchestra.run.smithi090.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:38.303 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:38.303 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T10:55:38.303 DEBUG:teuthology.parallel:result is None 2023-11-28T10:55:38.348 INFO:teuthology.orchestra.run.smithi090.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:38.394 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum reinstall -y kernel || true 2023-11-28T10:55:38.723 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:38.724 INFO:teuthology.orchestra.run.smithi090.stdout:Unable to read consumer identity 2023-11-28T10:55:38.985 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:36 AM UTC. 2023-11-28T10:55:39.030 INFO:teuthology.orchestra.run.smithi090.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:39.058 INFO:teuthology.orchestra.run.smithi090.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:39.103 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T10:55:39.184 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:39.184 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:39.184 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T10:55:39.184 DEBUG:teuthology.parallel:result is None 2023-11-28T10:55:39.185 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T10:55:39.192 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T10:55:39.192 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T10:55:39.195 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T10:55:39.213 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T10:55:39.219 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T10:55:39.225 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T10:55:39.225 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T10:55:39.255 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T10:55:39.278 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T10:55:39.285 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T10:55:39.286 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T10:55:39.316 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T10:55:39.353 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:39.359 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:39.363 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:39.398 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:39.400 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T10:55:39.407 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T10:55:39.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T10:55:39.410 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T10:55:39.439 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T10:55:39.447 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T10:55:39.447 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T10:55:39.481 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T10:55:39.497 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:39.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:39.630 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:39.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:39.757 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T10:55:39.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T10:55:39.825 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:39.867 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:39.920 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:39.998 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:40.045 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T10:55:40.045 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T10:55:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2023-11-28T10:55:40.113 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2023-11-28T10:55:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T10:55:40.213 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T10:55:40.444 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T10:55:40.452 INFO:teuthology.task.internal:Starting timer... 2023-11-28T10:55:40.452 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T10:55:40.468 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T10:55:40.477 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T10:55:40.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2023-11-28T10:55:40.540 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2023-11-28T10:55:40.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2023-11-28T10:55:40.604 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2023-11-28T10:55:40.606 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T10:55:40.606 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2023-11-28T10:55:40.627 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2023-11-28T10:55:40.628 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2023-11-28T10:55:40.652 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2023-11-28T10:55:40.652 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi090.front.sepia.ceph.com': 'permissive'} 2023-11-28T10:55:40.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T10:55:40.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:40.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T10:55:40.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:40.726 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T10:55:40.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2023-11-28T10:55:40.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2023-11-28T10:55:40.793 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T10:55:40.803 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T10:55:40.803 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T10:55:40.823 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T10:55:40.824 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi090.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:05:08.689 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com')] 2023-11-28T11:05:08.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-11-28T11:05:08.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:05:08.760 DEBUG:teuthology.orchestra.run.smithi083:> true 2023-11-28T11:05:08.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-11-28T11:05:08.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2023-11-28T11:05:08.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:05:08.942 DEBUG:teuthology.orchestra.run.smithi090:> true 2023-11-28T11:05:09.019 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2023-11-28T11:05:09.019 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:05:09.031 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:05:09.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:05:09.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:05:09.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:05:09.035 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:05:09.065 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:05:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:05:09.093 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:05:09.109 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:05:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2023-11-28T11:05:09.165 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2023-11-28T11:05:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:05:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2023-11-28T11:05:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:05:09.204 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2023-11-28T11:05:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:05:09.233 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:05:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:bash: ntpq: command not found 2023-11-28T11:05:09.282 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:05:09.282 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2023-11-28T11:05:09.282 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.282 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.282 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.282 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:bash: ntpq: command not found 2023-11-28T11:05:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:05:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2023-11-28T11:05:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:09.317 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:05:09.326 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:05:09.326 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:05:09.326 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:05:09.326 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:05:09.332 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:05:09.332 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:05:09.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:05:09.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:05:09.338 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:05:09.339 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:05:09.489 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:05:09.489 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:05:09.567 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:05:09.567 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:05:09.602 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:05:09.602 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:05:09.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:05:09.631 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:05:09.631 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:05:09.678 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:05:09.678 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:05:09.679 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:05:09.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:05:09.707 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:05:09.707 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:05:09.775 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:05:09.779 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:05:09.844 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2023-11-28T11:05:09.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2023-11-28T11:05:09.852 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:05:09.918 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2023-11-28T11:05:09.919 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2023-11-28T11:05:10.217 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:10.287 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:11.241 INFO:teuthology.orchestra.run.smithi083.stdout:44 files removed 2023-11-28T11:05:11.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:05:11.374 INFO:teuthology.orchestra.run.smithi090.stdout:44 files removed 2023-11-28T11:05:11.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:05:11.626 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:11.751 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:13.182 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 163 kB/s | 80 kB 00:00 2023-11-28T11:05:13.442 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 41 kB/s | 9.4 kB 00:00 2023-11-28T11:05:13.784 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2023-11-28T11:05:13.965 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 73 kB/s | 80 kB 00:01 2023-11-28T11:05:13.967 INFO:teuthology.orchestra.run.smithi083.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-11-28T11:05:14.315 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-11-28T11:05:14.568 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-11-28T11:05:14.711 INFO:teuthology.orchestra.run.smithi090.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-28T11:05:15.922 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-11-28T11:05:19.422 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 350 kB/s | 24 kB 00:00 2023-11-28T11:05:20.698 INFO:teuthology.orchestra.run.smithi090.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-11-28T11:05:21.345 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 2.2 MB/s | 16 MB 00:07 2023-11-28T11:05:25.093 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 190 kB/s | 24 kB 00:00 2023-11-28T11:05:26.020 INFO:teuthology.orchestra.run.smithi083.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-11-28T11:05:28.709 INFO:teuthology.orchestra.run.smithi090.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 53 MB 00:01 2023-11-28T11:05:34.312 INFO:teuthology.orchestra.run.smithi083.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-11-28T11:05:39.917 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:39.918 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:39.932 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:39.933 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:39.934 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:05:39.936 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:05:39.938 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:05:39.941 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:05:39.942 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:05:40.071 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout:=========================================================================================================================== 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout:=========================================================================================================================== 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:05:40.083 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:05:40.084 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:05:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-11-28T11:05:40.086 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:05:40.087 INFO:teuthology.orchestra.run.smithi090.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:05:40.088 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:05:40.089 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:05:40.090 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:05:40.090 INFO:teuthology.orchestra.run.smithi090.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:05:40.097 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:05:40.098 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:05:40.099 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:05:40.100 INFO:teuthology.orchestra.run.smithi090.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling module streams: 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql 10 2023-11-28T11:05:40.101 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:05:40.102 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-11-28T11:05:40.102 INFO:teuthology.orchestra.run.smithi090.stdout:=========================================================================================================================== 2023-11-28T11:05:40.102 INFO:teuthology.orchestra.run.smithi090.stdout:Install 128 Packages 2023-11-28T11:05:40.102 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 3 Packages 2023-11-28T11:05:40.102 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:05:40.102 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 203 M 2023-11-28T11:05:40.103 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-11-28T11:05:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 13 kB/s | 6.4 kB 00:00 2023-11-28T11:05:43.554 INFO:teuthology.orchestra.run.smithi090.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 315 kB/s | 845 kB 00:02 2023-11-28T11:05:43.936 INFO:teuthology.orchestra.run.smithi090.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 431 kB/s | 165 kB 00:00 2023-11-28T11:05:45.327 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:45.328 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:45.342 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:45.342 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:45.344 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:05:45.345 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:05:45.347 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:05:45.350 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:05:45.351 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:05:45.487 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-11-28T11:05:45.499 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================================================== 2023-11-28T11:05:45.499 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-11-28T11:05:45.499 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================================================== 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:05:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:05:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:05:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:05:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:05:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:05:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:05:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:05:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:05:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:05:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:05:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:05:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:05:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:05:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:05:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:05:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:05:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:05:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling module streams: 2023-11-28T11:05:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql 10 2023-11-28T11:05:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:05:45.515 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-11-28T11:05:45.515 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================================================== 2023-11-28T11:05:45.515 INFO:teuthology.orchestra.run.smithi083.stdout:Install 128 Packages 2023-11-28T11:05:45.515 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 3 Packages 2023-11-28T11:05:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:05:45.516 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 203 M 2023-11-28T11:05:45.516 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-11-28T11:05:45.855 INFO:teuthology.orchestra.run.smithi083.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-11-28T11:05:46.871 INFO:teuthology.orchestra.run.smithi083.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 831 kB/s | 845 kB 00:01 2023-11-28T11:05:47.005 INFO:teuthology.orchestra.run.smithi083.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.2 MB/s | 165 kB 00:00 2023-11-28T11:05:47.129 INFO:teuthology.orchestra.run.smithi090.stdout:(4/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 3.2 MB/s | 22 MB 00:06 2023-11-28T11:05:47.604 INFO:teuthology.orchestra.run.smithi090.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 3.3 MB/s | 1.5 MB 00:00 2023-11-28T11:05:48.146 INFO:teuthology.orchestra.run.smithi090.stdout:(6/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 553 kB/s | 2.3 MB 00:04 2023-11-28T11:05:48.364 INFO:teuthology.orchestra.run.smithi083.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-11-28T11:05:48.447 INFO:teuthology.orchestra.run.smithi083.stdout:(5/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-11-28T11:05:48.763 INFO:teuthology.orchestra.run.smithi090.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 4.0 MB/s | 4.6 MB 00:01 2023-11-28T11:05:49.339 INFO:teuthology.orchestra.run.smithi083.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 1.6 MB/s | 1.5 MB 00:00 2023-11-28T11:05:51.198 INFO:teuthology.orchestra.run.smithi083.stdout:(7/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 3.8 MB/s | 22 MB 00:05 2023-11-28T11:05:51.273 INFO:teuthology.orchestra.run.smithi083.stdout:(8/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 1.6 MB/s | 4.6 MB 00:02 2023-11-28T11:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 243 kB/s | 24 kB 00:00 2023-11-28T11:05:51.805 INFO:teuthology.orchestra.run.smithi090.stdout:(8/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el8 4.3 MB/s | 13 MB 00:03 2023-11-28T11:05:51.889 INFO:teuthology.orchestra.run.smithi090.stdout:(9/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 477 kB/s | 5.4 MB 00:11 2023-11-28T11:05:51.914 INFO:teuthology.orchestra.run.smithi090.stdout:(10/131): ceph-selinux-18.0.0-2697.g2f5f6028.el 224 kB/s | 24 kB 00:00 2023-11-28T11:05:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 642 kB/s | 257 kB 00:00 2023-11-28T11:05:52.423 INFO:teuthology.orchestra.run.smithi090.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 281 kB/s | 30 kB 00:00 2023-11-28T11:05:53.523 INFO:teuthology.orchestra.run.smithi090.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 698 kB/s | 767 kB 00:01 2023-11-28T11:05:53.792 INFO:teuthology.orchestra.run.smithi090.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 512 kB/s | 137 kB 00:00 2023-11-28T11:05:54.067 INFO:teuthology.orchestra.run.smithi090.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 505 kB/s | 139 kB 00:00 2023-11-28T11:05:54.648 INFO:teuthology.orchestra.run.smithi083.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 3.8 MB/s | 13 MB 00:03 2023-11-28T11:05:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 768 kB/s | 474 kB 00:00 2023-11-28T11:05:54.748 INFO:teuthology.orchestra.run.smithi083.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 2.5 MB/s | 257 kB 00:00 2023-11-28T11:05:54.849 INFO:teuthology.orchestra.run.smithi083.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 305 kB/s | 30 kB 00:00 2023-11-28T11:05:55.116 INFO:teuthology.orchestra.run.smithi083.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 2.8 MB/s | 767 kB 00:00 2023-11-28T11:05:55.207 INFO:teuthology.orchestra.run.smithi083.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 137 kB 00:00 2023-11-28T11:05:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.5 MB/s | 139 kB 00:00 2023-11-28T11:05:55.558 INFO:teuthology.orchestra.run.smithi083.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 1.8 MB/s | 474 kB 00:00 2023-11-28T11:05:57.550 INFO:teuthology.orchestra.run.smithi083.stdout:(17/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 2.2 MB/s | 18 MB 00:08 2023-11-28T11:05:57.634 INFO:teuthology.orchestra.run.smithi083.stdout:(18/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 3.1 MB/s | 6.5 MB 00:02 2023-11-28T11:05:57.659 INFO:teuthology.orchestra.run.smithi083.stdout:(19/131): python3-ceph-argparse-18.0.0-2697.g2f 411 kB/s | 45 kB 00:00 2023-11-28T11:05:57.709 INFO:teuthology.orchestra.run.smithi083.stdout:(20/131): python3-ceph-common-18.0.0-2697.g2f5f 1.5 MB/s | 117 kB 00:00 2023-11-28T11:05:57.818 INFO:teuthology.orchestra.run.smithi083.stdout:(21/131): python3-cephfs-18.0.0-2697.g2f5f6028. 1.2 MB/s | 190 kB 00:00 2023-11-28T11:05:57.860 INFO:teuthology.orchestra.run.smithi083.stdout:(22/131): python3-rados-18.0.0-2697.g2f5f6028.e 2.4 MB/s | 371 kB 00:00 2023-11-28T11:05:57.952 INFO:teuthology.orchestra.run.smithi083.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.1 MB/s | 107 kB 00:00 2023-11-28T11:05:57.977 INFO:teuthology.orchestra.run.smithi083.stdout:(24/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.2 MB/s | 364 kB 00:00 2023-11-28T11:05:58.027 INFO:teuthology.orchestra.run.smithi083.stdout:(25/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.2 MB/s | 94 kB 00:00 2023-11-28T11:05:58.119 INFO:teuthology.orchestra.run.smithi083.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.9 MB/s | 180 kB 00:00 2023-11-28T11:05:58.195 INFO:teuthology.orchestra.run.smithi083.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 307 kB/s | 23 kB 00:00 2023-11-28T11:05:58.287 INFO:teuthology.orchestra.run.smithi083.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.3 MB/s | 123 kB 00:00 2023-11-28T11:05:58.829 INFO:teuthology.orchestra.run.smithi083.stdout:(29/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 3.2 MB/s | 1.7 MB 00:00 2023-11-28T11:05:59.313 INFO:teuthology.orchestra.run.smithi083.stdout:(30/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 2.6 MB/s | 3.4 MB 00:01 2023-11-28T11:05:59.471 INFO:teuthology.orchestra.run.smithi083.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 1.5 MB/s | 236 kB 00:00 2023-11-28T11:05:59.571 INFO:teuthology.orchestra.run.smithi083.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 474 kB/s | 47 kB 00:00 2023-11-28T11:05:59.663 INFO:teuthology.orchestra.run.smithi083.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 158 kB/s | 15 kB 00:00 2023-11-28T11:05:59.847 INFO:teuthology.orchestra.run.smithi083.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 1.4 MB/s | 263 kB 00:00 2023-11-28T11:05:59.956 INFO:teuthology.orchestra.run.smithi083.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 765 kB/s | 83 kB 00:00 2023-11-28T11:06:00.048 INFO:teuthology.orchestra.run.smithi083.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 197 kB/s | 18 kB 00:00 2023-11-28T11:06:00.356 INFO:teuthology.orchestra.run.smithi083.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 993 kB/s | 306 kB 00:00 2023-11-28T11:06:00.415 INFO:teuthology.orchestra.run.smithi083.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-11-28T11:06:00.465 INFO:teuthology.orchestra.run.smithi083.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-11-28T11:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout:(17/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 1.1 MB/s | 6.5 MB 00:05 2023-11-28T11:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 383 kB/s | 45 kB 00:00 2023-11-28T11:06:00.816 INFO:teuthology.orchestra.run.smithi083.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 3.7 MB/s | 7.4 MB 00:01 2023-11-28T11:06:00.818 INFO:teuthology.orchestra.run.smithi090.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 639 kB/s | 117 kB 00:00 2023-11-28T11:06:00.841 INFO:teuthology.orchestra.run.smithi083.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-11-28T11:06:00.866 INFO:teuthology.orchestra.run.smithi083.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 872 kB/s | 44 kB 00:00 2023-11-28T11:06:00.917 INFO:teuthology.orchestra.run.smithi083.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 660 kB/s | 33 kB 00:00 2023-11-28T11:06:00.967 INFO:teuthology.orchestra.run.smithi083.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 331 kB/s | 16 kB 00:00 2023-11-28T11:06:01.002 INFO:teuthology.orchestra.run.smithi090.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 1.0 MB/s | 190 kB 00:00 2023-11-28T11:06:01.092 INFO:teuthology.orchestra.run.smithi083.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-11-28T11:06:01.126 INFO:teuthology.orchestra.run.smithi083.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 651 kB/s | 185 kB 00:00 2023-11-28T11:06:01.218 INFO:teuthology.orchestra.run.smithi083.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 886 kB/s | 81 kB 00:00 2023-11-28T11:06:01.268 INFO:teuthology.orchestra.run.smithi083.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-11-28T11:06:01.302 INFO:teuthology.orchestra.run.smithi090.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 1.2 MB/s | 371 kB 00:00 2023-11-28T11:06:01.335 INFO:teuthology.orchestra.run.smithi083.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-11-28T11:06:01.352 INFO:teuthology.orchestra.run.smithi083.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-11-28T11:06:01.402 INFO:teuthology.orchestra.run.smithi083.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 976 kB/s | 49 kB 00:00 2023-11-28T11:06:01.453 INFO:teuthology.orchestra.run.smithi083.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-11-28T11:06:01.503 INFO:teuthology.orchestra.run.smithi083.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-28T11:06:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 1.2 MB/s | 364 kB 00:00 2023-11-28T11:06:01.687 INFO:teuthology.orchestra.run.smithi083.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2023-11-28T11:06:01.720 INFO:teuthology.orchestra.run.smithi090.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 984 kB/s | 107 kB 00:00 2023-11-28T11:06:01.737 INFO:teuthology.orchestra.run.smithi083.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2023-11-28T11:06:01.796 INFO:teuthology.orchestra.run.smithi083.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 576 kB/s | 34 kB 00:00 2023-11-28T11:06:01.820 INFO:teuthology.orchestra.run.smithi090.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 946 kB/s | 94 kB 00:00 2023-11-28T11:06:01.921 INFO:teuthology.orchestra.run.smithi083.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-11-28T11:06:01.955 INFO:teuthology.orchestra.run.smithi083.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 1.9 MB/s | 1.2 MB 00:00 2023-11-28T11:06:01.980 INFO:teuthology.orchestra.run.smithi083.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-11-28T11:06:02.014 INFO:teuthology.orchestra.run.smithi083.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-11-28T11:06:02.039 INFO:teuthology.orchestra.run.smithi083.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 408 kB/s | 24 kB 00:00 2023-11-28T11:06:02.072 INFO:teuthology.orchestra.run.smithi083.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2023-11-28T11:06:02.098 INFO:teuthology.orchestra.run.smithi083.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 470 kB/s | 27 kB 00:00 2023-11-28T11:06:02.232 INFO:teuthology.orchestra.run.smithi083.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-11-28T11:06:02.257 INFO:teuthology.orchestra.run.smithi083.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-11-28T11:06:02.291 INFO:teuthology.orchestra.run.smithi083.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-28T11:06:02.349 INFO:teuthology.orchestra.run.smithi083.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 928 kB/s | 85 kB 00:00 2023-11-28T11:06:02.374 INFO:teuthology.orchestra.run.smithi083.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2023-11-28T11:06:02.404 INFO:teuthology.orchestra.run.smithi083.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:06:02.450 INFO:teuthology.orchestra.run.smithi083.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 938 kB/s | 47 kB 00:00 2023-11-28T11:06:02.492 INFO:teuthology.orchestra.run.smithi083.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-11-28T11:06:02.504 INFO:teuthology.orchestra.run.smithi090.stdout:(25/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 1.2 MB/s | 18 MB 00:14 2023-11-28T11:06:02.525 INFO:teuthology.orchestra.run.smithi083.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-11-28T11:06:02.559 INFO:teuthology.orchestra.run.smithi083.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-11-28T11:06:02.601 INFO:teuthology.orchestra.run.smithi083.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-11-28T11:06:02.612 INFO:teuthology.orchestra.run.smithi090.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.6 MB/s | 180 kB 00:00 2023-11-28T11:06:02.635 INFO:teuthology.orchestra.run.smithi083.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.6 MB/s | 87 kB 00:00 2023-11-28T11:06:02.668 INFO:teuthology.orchestra.run.smithi083.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.3 MB/s | 43 kB 00:00 2023-11-28T11:06:02.710 INFO:teuthology.orchestra.run.smithi083.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-11-28T11:06:02.721 INFO:teuthology.orchestra.run.smithi090.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 213 kB/s | 23 kB 00:00 2023-11-28T11:06:02.744 INFO:teuthology.orchestra.run.smithi083.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 6.7 MB/s | 229 kB 00:00 2023-11-28T11:06:02.795 INFO:teuthology.orchestra.run.smithi083.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-11-28T11:06:02.821 INFO:teuthology.orchestra.run.smithi090.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.2 MB/s | 123 kB 00:00 2023-11-28T11:06:02.828 INFO:teuthology.orchestra.run.smithi083.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 565 kB/s | 19 kB 00:00 2023-11-28T11:06:02.870 INFO:teuthology.orchestra.run.smithi083.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-11-28T11:06:02.912 INFO:teuthology.orchestra.run.smithi083.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2023-11-28T11:06:02.938 INFO:teuthology.orchestra.run.smithi083.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 716 kB/s | 18 kB 00:00 2023-11-28T11:06:02.988 INFO:teuthology.orchestra.run.smithi083.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-11-28T11:06:03.067 INFO:teuthology.orchestra.run.smithi083.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.0 MB/s | 238 kB 00:00 2023-11-28T11:06:03.155 INFO:teuthology.orchestra.run.smithi083.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 26 MB/s | 2.3 MB 00:00 2023-11-28T11:06:03.198 INFO:teuthology.orchestra.run.smithi083.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-11-28T11:06:03.239 INFO:teuthology.orchestra.run.smithi083.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-11-28T11:06:03.273 INFO:teuthology.orchestra.run.smithi083.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.8 MB/s | 60 kB 00:00 2023-11-28T11:06:03.297 INFO:teuthology.orchestra.run.smithi090.stdout:(29/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 3.6 MB/s | 1.7 MB 00:00 2023-11-28T11:06:03.307 INFO:teuthology.orchestra.run.smithi083.stdout:(90/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.9 MB/s | 98 kB 00:00 2023-11-28T11:06:03.349 INFO:teuthology.orchestra.run.smithi083.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2023-11-28T11:06:03.390 INFO:teuthology.orchestra.run.smithi083.stdout:(92/131): libcom_err-devel-1.45.6-4.el8.x86_64. 935 kB/s | 39 kB 00:00 2023-11-28T11:06:03.433 INFO:teuthology.orchestra.run.smithi083.stdout:(93/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 15 MB/s | 644 kB 00:00 2023-11-28T11:06:03.466 INFO:teuthology.orchestra.run.smithi083.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 5.1 MB/s | 171 kB 00:00 2023-11-28T11:06:03.500 INFO:teuthology.orchestra.run.smithi083.stdout:(95/131): NetworkManager-initscripts-updown-1.3 4.0 MB/s | 138 kB 00:00 2023-11-28T11:06:03.542 INFO:teuthology.orchestra.run.smithi083.stdout:(96/131): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2023-11-28T11:06:03.592 INFO:teuthology.orchestra.run.smithi083.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-11-28T11:06:03.635 INFO:teuthology.orchestra.run.smithi083.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-28T11:06:03.676 INFO:teuthology.orchestra.run.smithi083.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-28T11:06:03.718 INFO:teuthology.orchestra.run.smithi083.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-28T11:06:03.760 INFO:teuthology.orchestra.run.smithi083.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 936 kB/s | 39 kB 00:00 2023-11-28T11:06:03.810 INFO:teuthology.orchestra.run.smithi083.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-11-28T11:06:03.861 INFO:teuthology.orchestra.run.smithi083.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 10 MB/s | 538 kB 00:00 2023-11-28T11:06:03.994 INFO:teuthology.orchestra.run.smithi083.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-11-28T11:06:04.020 INFO:teuthology.orchestra.run.smithi083.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 990 kB/s | 157 kB 00:00 2023-11-28T11:06:04.045 INFO:teuthology.orchestra.run.smithi083.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-11-28T11:06:04.056 INFO:teuthology.orchestra.run.smithi090.stdout:(30/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 3.8 MB/s | 47 MB 00:12 2023-11-28T11:06:04.079 INFO:teuthology.orchestra.run.smithi083.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-28T11:06:04.129 INFO:teuthology.orchestra.run.smithi083.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 530 kB/s | 27 kB 00:00 2023-11-28T11:06:04.171 INFO:teuthology.orchestra.run.smithi083.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-28T11:06:04.255 INFO:teuthology.orchestra.run.smithi083.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-11-28T11:06:04.264 INFO:teuthology.orchestra.run.smithi090.stdout:(31/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 1.4 MB/s | 3.4 MB 00:02 2023-11-28T11:06:04.280 INFO:teuthology.orchestra.run.smithi083.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 405 kB/s | 44 kB 00:00 2023-11-28T11:06:04.290 INFO:teuthology.orchestra.run.smithi090.stdout:(32/131): ceph-mgr-modules-core-18.0.0-2697.g2f 1.0 MB/s | 236 kB 00:00 2023-11-28T11:06:04.330 INFO:teuthology.orchestra.run.smithi083.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-11-28T11:06:04.373 INFO:teuthology.orchestra.run.smithi090.stdout:(33/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 437 kB/s | 47 kB 00:00 2023-11-28T11:06:04.389 INFO:teuthology.orchestra.run.smithi083.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 28 MB/s | 3.7 MB 00:00 2023-11-28T11:06:04.399 INFO:teuthology.orchestra.run.smithi090.stdout:(34/131): ceph-prometheus-alerts-18.0.0-2697.g2 134 kB/s | 15 kB 00:00 2023-11-28T11:06:04.406 INFO:teuthology.orchestra.run.smithi083.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.9 MB/s | 225 kB 00:00 2023-11-28T11:06:04.432 INFO:teuthology.orchestra.run.smithi083.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-11-28T11:06:04.457 INFO:teuthology.orchestra.run.smithi083.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.4 MB/s | 126 kB 00:00 2023-11-28T11:06:04.482 INFO:teuthology.orchestra.run.smithi083.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-11-28T11:06:04.482 INFO:teuthology.orchestra.run.smithi090.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 993 kB/s | 83 kB 00:00 2023-11-28T11:06:04.507 INFO:teuthology.orchestra.run.smithi083.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-28T11:06:04.549 INFO:teuthology.orchestra.run.smithi083.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-28T11:06:04.574 INFO:teuthology.orchestra.run.smithi090.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 197 kB/s | 18 kB 00:00 2023-11-28T11:06:04.591 INFO:teuthology.orchestra.run.smithi083.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-28T11:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout:(37/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 982 kB/s | 263 kB 00:00 2023-11-28T11:06:04.683 INFO:teuthology.orchestra.run.smithi083.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 28 MB/s | 4.9 MB 00:00 2023-11-28T11:06:04.725 INFO:teuthology.orchestra.run.smithi083.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-28T11:06:04.784 INFO:teuthology.orchestra.run.smithi090.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 5.0 MB/s | 7.4 MB 00:01 2023-11-28T11:06:04.809 INFO:teuthology.orchestra.run.smithi083.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-11-28T11:06:04.834 INFO:teuthology.orchestra.run.smithi083.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-11-28T11:06:04.884 INFO:teuthology.orchestra.run.smithi090.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2023-11-28T11:06:04.884 INFO:teuthology.orchestra.run.smithi083.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-11-28T11:06:04.943 INFO:teuthology.orchestra.run.smithi090.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 476 kB/s | 75 kB 00:00 2023-11-28T11:06:04.968 INFO:teuthology.orchestra.run.smithi090.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 779 kB/s | 306 kB 00:00 2023-11-28T11:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout:(126/131): postgresql-server-10.21-2.module+el8 30 MB/s | 5.1 MB 00:00 2023-11-28T11:06:05.002 INFO:teuthology.orchestra.run.smithi090.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 750 kB/s | 44 kB 00:00 2023-11-28T11:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-11-28T11:06:05.027 INFO:teuthology.orchestra.run.smithi090.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-11-28T11:06:05.052 INFO:teuthology.orchestra.run.smithi090.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.8 MB/s | 476 kB 00:00 2023-11-28T11:06:05.077 INFO:teuthology.orchestra.run.smithi090.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 438 kB/s | 33 kB 00:00 2023-11-28T11:06:05.103 INFO:teuthology.orchestra.run.smithi090.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-11-28T11:06:05.145 INFO:teuthology.orchestra.run.smithi090.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-11-28T11:06:05.203 INFO:teuthology.orchestra.run.smithi090.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 810 kB/s | 81 kB 00:00 2023-11-28T11:06:05.229 INFO:teuthology.orchestra.run.smithi090.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-11-28T11:06:05.254 INFO:teuthology.orchestra.run.smithi090.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-11-28T11:06:05.279 INFO:teuthology.orchestra.run.smithi090.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2023-11-28T11:06:05.330 INFO:teuthology.orchestra.run.smithi090.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2023-11-28T11:06:05.355 INFO:teuthology.orchestra.run.smithi090.stdout:(53/131): python3-logutils-0.3.5-11.el8.noarch. 484 kB/s | 49 kB 00:00 2023-11-28T11:06:05.380 INFO:teuthology.orchestra.run.smithi090.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-28T11:06:05.414 INFO:teuthology.orchestra.run.smithi090.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-11-28T11:06:05.439 INFO:teuthology.orchestra.run.smithi090.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-11-28T11:06:05.481 INFO:teuthology.orchestra.run.smithi090.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2023-11-28T11:06:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 364 kB/s | 34 kB 00:00 2023-11-28T11:06:05.532 INFO:teuthology.orchestra.run.smithi090.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-11-28T11:06:05.557 INFO:teuthology.orchestra.run.smithi090.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 850 kB/s | 64 kB 00:00 2023-11-28T11:06:05.574 INFO:teuthology.orchestra.run.smithi090.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 278 kB/s | 19 kB 00:00 2023-11-28T11:06:05.591 INFO:teuthology.orchestra.run.smithi090.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 406 kB/s | 24 kB 00:00 2023-11-28T11:06:05.602 INFO:teuthology.orchestra.run.smithi083.stdout:(128/131): ceph-test-18.0.0-2697.g2f5f6028.el8. 3.3 MB/s | 47 MB 00:14 2023-11-28T11:06:05.618 INFO:teuthology.orchestra.run.smithi090.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 487 kB/s | 29 kB 00:00 2023-11-28T11:06:05.643 INFO:teuthology.orchestra.run.smithi090.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 402 kB/s | 27 kB 00:00 2023-11-28T11:06:05.652 INFO:teuthology.orchestra.run.smithi083.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-28T11:06:05.677 INFO:teuthology.orchestra.run.smithi090.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.9 MB/s | 255 kB 00:00 2023-11-28T11:06:05.702 INFO:teuthology.orchestra.run.smithi090.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-28T11:06:05.736 INFO:teuthology.orchestra.run.smithi090.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-11-28T11:06:05.761 INFO:teuthology.orchestra.run.smithi090.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-11-28T11:06:05.786 INFO:teuthology.orchestra.run.smithi090.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:06:05.869 INFO:teuthology.orchestra.run.smithi083.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 4.2 MB/s | 3.6 MB 00:00 2023-11-28T11:06:05.903 INFO:teuthology.orchestra.run.smithi090.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2023-11-28T11:06:05.962 INFO:teuthology.orchestra.run.smithi090.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-11-28T11:06:06.004 INFO:teuthology.orchestra.run.smithi090.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-28T11:06:06.046 INFO:teuthology.orchestra.run.smithi090.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-28T11:06:06.096 INFO:teuthology.orchestra.run.smithi090.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-28T11:06:06.130 INFO:teuthology.orchestra.run.smithi090.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-11-28T11:06:06.269 INFO:teuthology.orchestra.run.smithi083.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 2.8 MB/s | 3.7 MB 00:01 2023-11-28T11:06:06.274 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:06:06.274 INFO:teuthology.orchestra.run.smithi083.stdout:Total 9.8 MB/s | 203 MB 00:20 2023-11-28T11:06:06.297 INFO:teuthology.orchestra.run.smithi090.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 27 MB/s | 14 MB 00:00 2023-11-28T11:06:06.339 INFO:teuthology.orchestra.run.smithi090.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.2 MB/s | 688 kB 00:00 2023-11-28T11:06:06.381 INFO:teuthology.orchestra.run.smithi090.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.3 MB/s | 538 kB 00:00 2023-11-28T11:06:06.406 INFO:teuthology.orchestra.run.smithi090.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-11-28T11:06:06.431 INFO:teuthology.orchestra.run.smithi090.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-11-28T11:06:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-11-28T11:06:06.498 INFO:teuthology.orchestra.run.smithi090.stdout:(82/131): python3-webtest-2.0.33-1.el8.noarch.r 104 kB/s | 85 kB 00:00 2023-11-28T11:06:06.524 INFO:teuthology.orchestra.run.smithi090.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 526 kB/s | 27 kB 00:00 2023-11-28T11:06:06.566 INFO:teuthology.orchestra.run.smithi090.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.2 MB/s | 44 kB 00:00 2023-11-28T11:06:06.599 INFO:teuthology.orchestra.run.smithi090.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-11-28T11:06:06.642 INFO:teuthology.orchestra.run.smithi090.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-11-28T11:06:06.684 INFO:teuthology.orchestra.run.smithi090.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-28T11:06:06.759 INFO:teuthology.orchestra.run.smithi090.stdout:(88/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-11-28T11:06:06.785 INFO:teuthology.orchestra.run.smithi090.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 631 kB/s | 64 kB 00:00 2023-11-28T11:06:06.819 INFO:teuthology.orchestra.run.smithi090.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2023-11-28T11:06:06.878 INFO:teuthology.orchestra.run.smithi090.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 217 kB/s | 20 kB 00:00 2023-11-28T11:06:06.904 INFO:teuthology.orchestra.run.smithi090.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.2 MB/s | 103 kB 00:00 2023-11-28T11:06:06.978 INFO:teuthology.orchestra.run.smithi090.stdout:(93/131): python3-numpy-1.14.3-10.el8.x86_64.rp 9.1 MB/s | 3.7 MB 00:00 2023-11-28T11:06:07.012 INFO:teuthology.orchestra.run.smithi090.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-11-28T11:06:07.037 INFO:teuthology.orchestra.run.smithi090.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-11-28T11:06:07.079 INFO:teuthology.orchestra.run.smithi090.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-28T11:06:07.121 INFO:teuthology.orchestra.run.smithi090.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-28T11:06:07.133 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-11-28T11:06:07.213 INFO:teuthology.orchestra.run.smithi090.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-11-28T11:06:07.253 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-11-28T11:06:07.253 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-11-28T11:06:07.372 INFO:teuthology.orchestra.run.smithi090.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-11-28T11:06:07.555 INFO:teuthology.orchestra.run.smithi090.stdout:(100/131): postgresql-contrib-10.21-2.module+el 2.3 MB/s | 808 kB 00:00 2023-11-28T11:06:07.589 INFO:teuthology.orchestra.run.smithi090.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-11-28T11:06:07.640 INFO:teuthology.orchestra.run.smithi090.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 5.6 MB/s | 1.5 MB 00:00 2023-11-28T11:06:07.673 INFO:teuthology.orchestra.run.smithi090.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2023-11-28T11:06:07.699 INFO:teuthology.orchestra.run.smithi090.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 727 kB/s | 43 kB 00:00 2023-11-28T11:06:07.724 INFO:teuthology.orchestra.run.smithi090.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-28T11:06:07.749 INFO:teuthology.orchestra.run.smithi090.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 959 kB/s | 48 kB 00:00 2023-11-28T11:06:07.783 INFO:teuthology.orchestra.run.smithi090.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-11-28T11:06:07.808 INFO:teuthology.orchestra.run.smithi090.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 727 kB/s | 43 kB 00:00 2023-11-28T11:06:07.833 INFO:teuthology.orchestra.run.smithi090.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-28T11:06:07.867 INFO:teuthology.orchestra.run.smithi090.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-11-28T11:06:07.909 INFO:teuthology.orchestra.run.smithi090.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-11-28T11:06:07.926 INFO:teuthology.orchestra.run.smithi090.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-11-28T11:06:07.960 INFO:teuthology.orchestra.run.smithi090.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-28T11:06:08.035 INFO:teuthology.orchestra.run.smithi090.stdout:(114/131): postgresql-server-10.21-2.module+el8 5.5 MB/s | 5.1 MB 00:00 2023-11-28T11:06:08.062 INFO:teuthology.orchestra.run.smithi090.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 176 kB/s | 18 kB 00:00 2023-11-28T11:06:08.104 INFO:teuthology.orchestra.run.smithi090.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.1 MB/s | 559 kB 00:00 2023-11-28T11:06:08.129 INFO:teuthology.orchestra.run.smithi090.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-28T11:06:08.162 INFO:teuthology.orchestra.run.smithi090.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.6 MB/s | 605 kB 00:00 2023-11-28T11:06:08.188 INFO:teuthology.orchestra.run.smithi090.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-11-28T11:06:08.213 INFO:teuthology.orchestra.run.smithi090.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-28T11:06:08.239 INFO:teuthology.orchestra.run.smithi090.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-28T11:06:08.264 INFO:teuthology.orchestra.run.smithi090.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-28T11:06:08.290 INFO:teuthology.orchestra.run.smithi090.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-11-28T11:06:08.315 INFO:teuthology.orchestra.run.smithi090.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-11-28T11:06:08.349 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-11-28T11:06:08.353 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-11-28T11:06:08.356 INFO:teuthology.orchestra.run.smithi090.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-11-28T11:06:08.391 INFO:teuthology.orchestra.run.smithi090.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-11-28T11:06:08.424 INFO:teuthology.orchestra.run.smithi090.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.0 MB/s | 138 kB 00:00 2023-11-28T11:06:08.491 INFO:teuthology.orchestra.run.smithi090.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-11-28T11:06:08.550 INFO:teuthology.orchestra.run.smithi090.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-28T11:06:08.875 INFO:teuthology.orchestra.run.smithi090.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 7.6 MB/s | 3.7 MB 00:00 2023-11-28T11:06:09.692 INFO:teuthology.orchestra.run.smithi090.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 2.8 MB/s | 3.6 MB 00:01 2023-11-28T11:06:09.697 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:06:09.697 INFO:teuthology.orchestra.run.smithi090.stdout:Total 6.8 MB/s | 203 MB 00:29 2023-11-28T11:06:10.506 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-11-28T11:06:10.626 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-11-28T11:06:10.626 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-11-28T11:06:11.770 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-11-28T11:06:11.774 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-11-28T11:06:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-11-28T11:06:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:06:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:06:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:06:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:06:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:06:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:06:14.928 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:06:15.323 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:06:15.502 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:15.640 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:15.827 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:15.918 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:16.099 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:16.302 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T11:06:16.574 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:06:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-11-28T11:06:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:06:16.925 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:06:17.114 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:06:17.188 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:06:17.326 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T11:06:17.494 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:06:17.626 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:06:17.680 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:06:17.845 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:17.869 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:17.974 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:18.176 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:18.232 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:18.494 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:18.514 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T11:06:18.699 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T11:06:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T11:06:18.894 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:18.984 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:06:19.078 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:19.122 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:06:19.212 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:06:19.394 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:19.493 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:19.707 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:06:19.877 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:06:19.963 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:06:20.035 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:06:20.147 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:06:20.598 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:06:20.635 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:06:20.814 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:06:20.826 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:06:21.019 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:21.130 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:21.327 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:21.413 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:21.530 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:06:21.544 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:21.687 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:21.819 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:21.824 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T11:06:22.008 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T11:06:22.210 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:06:22.399 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:06:22.424 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:22.537 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:06:22.586 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:22.780 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:06:22.984 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:06:23.182 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:23.402 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:24.125 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:24.355 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T11:06:24.595 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T11:06:24.764 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T11:06:24.776 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:06:24.925 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T11:06:25.118 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T11:06:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:06:25.501 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:06:25.646 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:06:25.779 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T11:06:26.072 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T11:06:26.377 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T11:06:26.538 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:06:26.604 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T11:06:26.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:06:26.859 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T11:06:27.019 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:06:27.096 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T11:06:27.215 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:06:27.328 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T11:06:27.365 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:06:27.507 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T11:06:27.519 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T11:06:27.842 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:06:27.909 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:06:28.127 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T11:06:28.329 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T11:06:28.542 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:06:28.622 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:06:28.846 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T11:06:29.029 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T11:06:29.205 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T11:06:29.301 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T11:06:29.561 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T11:06:29.613 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T11:06:29.704 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T11:06:29.831 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:06:29.888 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T11:06:30.063 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:06:30.070 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T11:06:30.249 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:06:30.398 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:06:30.432 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T11:06:30.575 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T11:06:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T11:06:30.753 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T11:06:30.921 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T11:06:30.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T11:06:31.080 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T11:06:31.254 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T11:06:31.274 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T11:06:31.481 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T11:06:31.487 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T11:06:31.791 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T11:06:31.828 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T11:06:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T11:06:32.089 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T11:06:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T11:06:32.280 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T11:06:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T11:06:32.483 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T11:06:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T11:06:32.690 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T11:06:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T11:06:32.995 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T11:06:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T11:06:33.218 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T11:06:33.354 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T11:06:33.415 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T11:06:33.579 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T11:06:33.646 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T11:06:33.826 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T11:06:33.864 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T11:06:33.990 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:06:34.046 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T11:06:34.177 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:06:34.353 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T11:06:34.460 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:06:34.605 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T11:06:34.664 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:06:34.812 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:06:34.836 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T11:06:34.951 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:06:35.033 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T11:06:35.126 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:06:35.217 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T11:06:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:06:35.401 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T11:06:35.536 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:06:35.577 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:35.647 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:35.677 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:06:35.805 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:06:35.905 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T11:06:36.045 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:06:36.096 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T11:06:36.244 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:06:36.260 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:36.327 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:36.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:06:36.543 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T11:06:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:06:36.702 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T11:06:36.868 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:06:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:06:37.054 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:06:37.081 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:06:37.273 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:06:37.311 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:06:37.533 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:06:37.723 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:06:37.895 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:06:38.084 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:06:38.262 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:06:38.350 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:06:38.437 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:06:38.597 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:06:38.644 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:06:38.733 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:06:38.898 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:06:38.941 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:06:39.164 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:06:39.181 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:06:39.349 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:06:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:06:39.624 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:06:39.632 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:06:39.834 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:06:40.017 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:06:40.231 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:06:40.437 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:06:40.623 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:40.698 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:06:41.286 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:06:41.631 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:06:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:06:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:06:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:06:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:06:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:06:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:43.836 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:06:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:44.765 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:06:47.427 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:47.627 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:48.233 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:48.482 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:48.565 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:48.726 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:07:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:07:04.355 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /dev 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /run 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2023-11-28T11:07:04.356 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2023-11-28T11:07:04.357 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /dev 2023-11-28T11:07:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:06.267 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:06.812 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:07.462 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:07.479 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:08.069 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:07:08.069 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2023-11-28T11:07:08.069 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2023-11-28T11:07:08.069 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /dev 2023-11-28T11:07:08.069 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /run 2023-11-28T11:07:08.069 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2023-11-28T11:07:08.070 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2023-11-28T11:07:08.070 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2023-11-28T11:07:08.070 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2023-11-28T11:07:08.070 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /dev 2023-11-28T11:07:08.070 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:08.377 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:09.173 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:09.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:09.459 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:07:09.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:07:09.802 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:09.992 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:10.022 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:10.173 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:10.357 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:10.391 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:10.543 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:10.997 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:11.226 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:11.243 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:11.326 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:11.469 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:11.559 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:07:12.123 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:12.158 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:12.347 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:07:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:07:13.558 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:07:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:14.046 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:15.004 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:15.200 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:15.289 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:15.625 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:07:15.815 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:15.850 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:16.034 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:16.045 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:17.338 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:07:17.552 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:17.586 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:17.759 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:17.801 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:17.951 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:18.295 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:18.467 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:19.203 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:20.065 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:07:20.379 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T11:07:20.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T11:07:20.798 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T11:07:20.968 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:07:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:07:21.397 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T11:07:21.563 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:07:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:22.921 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:23.192 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:07:23.333 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:23.931 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:24.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:24.376 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:07:24.426 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T11:07:24.651 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:07:24.771 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T11:07:24.917 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:07:24.963 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T11:07:25.162 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:25.248 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:07:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:07:25.417 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:07:25.497 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:07:25.501 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:25.700 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:07:25.713 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:07:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:07:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:07:26.011 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T11:07:26.011 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:26.409 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:26.616 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:27.126 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:27.126 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:07:27.127 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:07:27.128 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:07:27.129 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:07:27.129 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:07:27.129 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:07:27.129 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:07:27.129 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:07:27.129 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:07:27.141 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:07:27.141 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:07:27.141 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:07:27.141 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:07:27.141 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:07:27.142 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:07:27.143 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:07:27.144 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T11:07:27.145 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T11:07:27.146 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T11:07:27.147 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T11:07:27.148 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T11:07:27.149 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:27.150 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:07:27.630 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:27.892 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:27.910 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:28.088 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:07:28.392 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:07:28.708 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:07:29.010 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:07:29.158 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:29.167 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:29.343 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T11:07:29.533 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:29.611 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:29.858 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:29.871 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:29.871 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:29.893 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:29.893 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:56 ago on Tue 28 Nov 2023 11:05:33 AM UTC. 2023-11-28T11:07:30.086 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:30.086 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:30.499 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:30.980 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:07:30.981 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:07:30.982 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:07:30.983 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:07:30.983 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:07:30.983 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:07:30.983 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:07:30.983 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:07:30.992 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:07:30.992 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:07:30.992 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:07:30.992 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:07:30.992 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:07:30.992 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:07:30.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:07:30.994 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:07:30.995 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T11:07:30.996 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T11:07:30.997 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T11:07:30.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:30.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:31.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:07:32.555 INFO:teuthology.orchestra.run.smithi083.stdout:Installed products updated. 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-11-28T11:07:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:07:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:07:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:07:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:07:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:07:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:07:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:07:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:07:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:32.803 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-11-28T11:07:32.971 DEBUG:teuthology.parallel:result is None 2023-11-28T11:07:33.667 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:33.668 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:05 ago on Tue 28 Nov 2023 11:05:28 AM UTC. 2023-11-28T11:07:36.176 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-11-28T11:07:36.414 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:36.414 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2023-11-28T11:07:36.415 INFO:teuthology.orchestra.run.smithi090.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:07:36.415 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.415 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.415 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-11-28T11:07:36.415 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:07:36.415 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:07:36.416 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:07:36.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.417 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:07:36.419 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.420 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.421 INFO:teuthology.orchestra.run.smithi090.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:07:36.422 INFO:teuthology.orchestra.run.smithi090.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.423 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:07:36.424 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:07:36.425 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:07:36.426 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.427 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:36.428 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-11-28T11:07:36.600 DEBUG:teuthology.parallel:result is None 2023-11-28T11:07:36.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:07:36.746 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:07:36.774 INFO:teuthology.orchestra.run.smithi083.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:36.775 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:36.775 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:07:36.775 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:07:36.916 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:07:36.939 INFO:teuthology.orchestra.run.smithi090.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:36.939 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:36.939 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:07:36.940 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T11:07:36.940 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:36.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:07:36.987 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:36.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:07:37.027 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T11:07:37.027 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:37.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:07:37.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:07:37.128 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:37.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:07:37.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:07:37.222 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T11:07:37.222 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:37.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:07:37.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:07:37.321 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:37.321 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:07:37.348 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:07:37.413 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T11:07:37.414 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:37.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:07:37.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:07:37.511 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:37.511 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:07:37.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:07:37.603 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T11:07:37.671 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T11:07:37.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2023-11-28T11:07:37.674 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /var/log/ceph 2023-11-28T11:07:37.699 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T11:07:37.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:07:37.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:07:37.767 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T11:07:37.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:07:37.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:07:37.842 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T11:07:37.842 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T11:07:37.842 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T11:07:37.842 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:37.842 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:07:37.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:38.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:38.052 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:38.053 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:07:38.070 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:38.190 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:38.221 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T11:07:38.221 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T11:07:38.222 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469953', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9437, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi090.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_10:48:03', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T11:07:38.222 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:07:38.224 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:07:38.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:07:38.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:07:38.340 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:38.340 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:07:38.362 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:07:38.362 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2023-11-28T11:07:38.419 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:07:38.419 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:38.419 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 392640 Links: 1 2023-11-28T11:07:38.420 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:38.420 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:38.420 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-28 11:07:26.831742899 +0000 2023-11-28T11:07:38.420 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-28 11:04:36.248513800 +0000 2023-11-28T11:07:38.420 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-28 11:04:36.248513800 +0000 2023-11-28T11:07:38.420 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-28T11:07:38.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:07:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-28T11:07:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-28T11:07:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000229653 s, 2.2 MB/s 2023-11-28T11:07:38.485 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:07:38.544 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2023-11-28T11:07:38.601 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:07:38.601 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:38.601 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 382599 Links: 1 2023-11-28T11:07:38.601 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:38.602 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:38.602 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-28 11:07:26.831742899 +0000 2023-11-28T11:07:38.602 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-28 11:04:36.815500173 +0000 2023-11-28T11:07:38.602 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-28 11:04:36.815500173 +0000 2023-11-28T11:07:38.602 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-28T11:07:38.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:07:38.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-28T11:07:38.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-28T11:07:38.666 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000201895 s, 2.5 MB/s 2023-11-28T11:07:38.667 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:07:38.726 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2023-11-28T11:07:38.782 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 392817 Links: 1 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-28 11:07:26.831742899 +0000 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-28 11:04:37.364486980 +0000 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-28 11:04:37.364486980 +0000 2023-11-28T11:07:38.783 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-28T11:07:38.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:07:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-28T11:07:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-28T11:07:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000214278 s, 2.4 MB/s 2023-11-28T11:07:38.849 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:07:38.907 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2023-11-28T11:07:38.964 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 390862 Links: 1 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-28 11:07:26.831742899 +0000 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-28 11:04:37.916473713 +0000 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-28 11:04:37.916473713 +0000 2023-11-28T11:07:38.965 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-28T11:07:38.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:07:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-28T11:07:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-28T11:07:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000186193 s, 2.7 MB/s 2023-11-28T11:07:39.031 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:07:39.088 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:39.088 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:39.089 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:07:39.109 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:07:39.109 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2023-11-28T11:07:39.169 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:07:39.169 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 383597 Links: 1 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2023-11-28 11:07:30.690367186 +0000 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2023-11-28 11:04:36.236196006 +0000 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2023-11-28 11:04:36.236196006 +0000 2023-11-28T11:07:39.170 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2023-11-28T11:07:39.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:07:39.239 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2023-11-28T11:07:39.239 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2023-11-28T11:07:39.239 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000281562 s, 1.8 MB/s 2023-11-28T11:07:39.239 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:07:39.299 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2023-11-28T11:07:39.357 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:07:39.357 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 393286 Links: 1 2023-11-28T11:07:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2023-11-28 11:07:30.690367186 +0000 2023-11-28T11:07:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2023-11-28 11:04:36.777187362 +0000 2023-11-28T11:07:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2023-11-28 11:04:36.777187362 +0000 2023-11-28T11:07:39.358 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2023-11-28T11:07:39.358 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:07:39.424 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2023-11-28T11:07:39.424 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2023-11-28T11:07:39.424 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000225886 s, 2.3 MB/s 2023-11-28T11:07:39.425 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:07:39.484 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2023-11-28T11:07:39.542 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:07:39.542 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:39.542 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 393425 Links: 1 2023-11-28T11:07:39.542 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:39.542 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:39.543 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2023-11-28 11:07:30.690367186 +0000 2023-11-28T11:07:39.543 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2023-11-28 11:04:37.344178302 +0000 2023-11-28T11:07:39.543 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2023-11-28 11:04:37.344178302 +0000 2023-11-28T11:07:39.543 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2023-11-28T11:07:39.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:07:39.608 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2023-11-28T11:07:39.608 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2023-11-28T11:07:39.608 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.00020419 s, 2.5 MB/s 2023-11-28T11:07:39.609 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:07:39.667 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 383715 Links: 1 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2023-11-28 11:07:30.690367186 +0000 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2023-11-28 11:04:37.865169977 +0000 2023-11-28T11:07:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2023-11-28 11:04:37.865169977 +0000 2023-11-28T11:07:39.726 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2023-11-28T11:07:39.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:07:39.791 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2023-11-28T11:07:39.791 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2023-11-28T11:07:39.791 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000191804 s, 2.7 MB/s 2023-11-28T11:07:39.792 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:07:39.850 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:39.850 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:07:39.850 INFO:tasks.ceph:Generating config... 2023-11-28T11:07:39.851 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T11:07:39.852 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T11:07:39.852 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T11:07:39.852 INFO:tasks.ceph:[client] pid file = 2023-11-28T11:07:39.852 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T11:07:39.852 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T11:07:39.852 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T11:07:39.852 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T11:07:39.852 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T11:07:39.852 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T11:07:39.852 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T11:07:39.852 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T11:07:39.852 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T11:07:39.853 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T11:07:39.854 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T11:07:39.855 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T11:07:39.855 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T11:07:39.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T11:07:39.915 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T11:07:39.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T11:07:39.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:39.994 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.b', '172.21.15.90'), ('mon.c', '[v2:172.21.15.90:3301,v1:172.21.15.90:6790]')] 2023-11-28T11:07:39.995 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,172.21.15.90,[v2:172.21.15.90:3301,v1:172.21.15.90:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T11:07:39.995 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:39.996 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:07:40.051 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --add b 172.21.15.90 --addv c '[v2:172.21.15.90:3301,v1:172.21.15.90:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:40.139 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:40.139 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid 69424f07-94c3-4dbc-8507-3551775e0a09 2023-11-28T11:07:40.139 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = pacific 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 69424f07-94c3-4dbc-8507-3551775e0a09 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2023-11-28T11:07:40.139412+0000 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:created 2023-11-28T11:07:40.139412+0000 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 16 (pacific) 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2023-11-28T11:07:40.140 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] mon.c 2023-11-28T11:07:40.141 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T11:07:40.142 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:07:40.159 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 69424f07-94c3-4dbc-8507-3551775e0a09... 2023-11-28T11:07:40.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:07:40.201 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.244 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2023-11-28T11:07:40.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:07:40.246 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2023-11-28T11:07:40.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.248 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2023-11-28T11:07:40.248 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2023-11-28T11:07:40.248 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2023-11-28T11:07:40.248 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.90,[v2:172.21.15.90:3301,v1:172.21.15.90:6790]" 2023-11-28T11:07:40.248 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on skipped message = False 2023-11-28T11:07:40.248 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 69424f07-94c3-4dbc-8507-3551775e0a09 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2023-11-28T11:07:40.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.250 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2023-11-28T11:07:40.251 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 20 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 20 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:07:40.252 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:40.253 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2023-11-28T11:07:40.254 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.255 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2023-11-28T11:07:40.256 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2023-11-28T11:07:40.257 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2023-11-28T11:07:40.258 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2023-11-28T11:07:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2023-11-28T11:07:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2023-11-28T11:07:40.259 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: chdir = "" 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: auth supported = cephx 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: filestore xattr use omap = true 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: mon clock drift allowed = 1.000 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.264 INFO:teuthology.orchestra.run.smithi090.stdout: osd crush chooseleaf type = 0 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: auth debug = true 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on old message = true 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on bug = true 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: mon pg warn max object skew = 0 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.265 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default size = 2 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow primary affinity = true 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow pg remap = true 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on no sortbitwise = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on too few osds = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: mon_allow_pool_size_one = true 2023-11-28T11:07:40.266 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: osd default data pool replay window = 5 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: mon allow pool delete = true 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: mon cluster log file level = debug 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: debug asserts on shutdown = true 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: mon health detail to clog = false 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: mon host = "172.21.15.83,172.21.15.90,[v2:172.21.15.90:3301,v1:172.21.15.90:6790]" 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on skipped message = False 2023-11-28T11:07:40.267 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 69424f07-94c3-4dbc-8507-3551775e0a09 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout:[osd] 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd journal size = 100 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub load threshold = 5.0 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub max interval = 600 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd recover clone overlap = true 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd recovery max chunk = 1048576 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug shutdown = true 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug op order = true 2023-11-28T11:07:40.268 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug verify stray on activate = true 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: osd open classes on start = true 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug pg log writeout = true 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: osd map max advance = 10 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: journal zero on create = true 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:07:40.269 INFO:teuthology.orchestra.run.smithi090.stdout: filestore apply finisher threads = 3 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: bdev debug aio = true 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug misdirected ops = true 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore block size = 96636764160 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore fsck on mount = True 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluefs = 20 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluestore = 20 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: debug osd = 20 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: debug rocksdb = 10 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:40.270 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd full ratio = 0.9 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: osd objectstore = bluestore 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: osd op complaint time = 180 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr] 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: debug mgr = 20 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:40.271 INFO:teuthology.orchestra.run.smithi090.stdout: mgr/telemetry/nag = false 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout:[mon] 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: debug paxos = 20 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: mon data avail warn = 5 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: mon mgr mkfs grace = 240 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd reporter subtree level = osd 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd prime pg temp = true 2023-11-28T11:07:40.272 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: # 1m isn't quite enough 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:07:40.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: mon op complaint time = 120 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout:[client] 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: rgw cache enabled = true 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable ops log = true 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable usage log = true 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: client mount timeout = 600 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: debug client = 20 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2023-11-28T11:07:40.274 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = "" 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout: rados mon op timeout = 900 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout: rados osd op timeout = 900 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.a] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.b] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.c] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout:[mds] 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout: debug mds = 20 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout: debug mds balancer = 20 2023-11-28T11:07:40.275 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: mds debug frag = True 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: mds debug scatterstat = True 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: mds op complaint time = 180 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: mds verify scatter = True 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: osd op complaint time = 180 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: rados mon op timeout = 900 2023-11-28T11:07:40.276 INFO:teuthology.orchestra.run.smithi090.stdout: rados osd op timeout = 900 2023-11-28T11:07:40.276 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T11:07:40.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T11:07:40.361 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T11:07:40.361 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:40.361 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T11:07:40.378 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:40.378 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T11:07:40.434 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:07:40.434 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:40.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:07:40.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:40.511 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:07:40.511 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:40.568 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:07:40.568 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:40.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:07:40.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:40.604 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:07:40.605 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:40.660 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T11:07:40.661 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T11:07:40.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:07:40.713 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:07:40.715 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:07:40.775 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:07:40.777 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T11:07:40.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:07:40.831 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:07:40.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T11:07:40.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:07:40.953 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:07:40.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T11:07:40.981 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:07:41.035 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:07:41.037 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T11:07:41.063 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:07:41.156 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:07:41.158 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T11:07:41.184 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T11:07:41.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T11:07:41.228 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T11:07:41.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T11:07:41.323 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T11:07:41.335 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T11:07:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T11:07:41.390 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T11:07:41.473 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T11:07:41.486 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T11:07:41.486 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:07:41.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:41.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:41.512 INFO:tasks.ceph:role: osd.0 2023-11-28T11:07:41.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:07:41.513 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:41.586 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:42.021 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-28T11:07:42.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-28T11:07:42.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:42.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:42.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:42.294 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:42.294 INFO:tasks.ceph:role: osd.1 2023-11-28T11:07:42.294 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:07:42.295 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:07:42.370 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:42.371 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:42.809 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-28T11:07:42.811 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-28T11:07:42.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:42.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:42.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:43.008 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:43.009 INFO:tasks.ceph:role: osd.2 2023-11-28T11:07:43.009 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:07:43.009 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:07:43.083 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:43.083 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:43.083 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:43.083 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:43.084 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:43.084 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:43.084 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:43.084 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:43.084 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:43.084 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:43.530 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-28T11:07:43.532 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-28T11:07:43.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:43.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:43.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:43.809 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:43.809 INFO:tasks.ceph:role: osd.3 2023-11-28T11:07:43.809 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:07:43.809 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:43.884 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:43.884 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:43.884 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:43.884 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:44.320 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-28T11:07:44.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-28T11:07:44.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:44.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:44.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:44.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:44.636+0000 7fb1025ee640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T11:07:44.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:44.636+0000 7fb1025ee640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T11:07:44.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:44.636+0000 7fb1025ee640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:44.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:44.636+0000 7fb1025ee640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:44.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:44.636+0000 7fb1025ee640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:44.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:44.636+0000 7fb1025ee640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T11:07:47.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:47.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:47.621+0000 7f46ced25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T11:07:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:47.622+0000 7f46ced25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T11:07:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:47.622+0000 7f46ced25640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:47.622+0000 7f46ced25640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:47.622+0000 7f46ced25640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:47.622+0000 7f46ced25640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T11:07:50.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:50.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:50.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:50.608+0000 7f92edde0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T11:07:50.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:50.609+0000 7f92edde0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T11:07:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:50.609+0000 7f92edde0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:07:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:50.609+0000 7f92edde0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:07:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:50.609+0000 7f92edde0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:07:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:50.609+0000 7f92edde0640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T11:07:53.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:53.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:53.605+0000 7fe14d87a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T11:07:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:53.606+0000 7fe14d87a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T11:07:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:53.606+0000 7fe14d87a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:07:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:53.606+0000 7fe14d87a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:07:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:53.606+0000 7fe14d87a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:07:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:07:53.606+0000 7fe14d87a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T11:07:56.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:56.500 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:56.532 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:56.533 INFO:tasks.ceph:role: osd.4 2023-11-28T11:07:56.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:07:56.533 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:56.611 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:57.047 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2023-11-28T11:07:57.050 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2023-11-28T11:07:57.050 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:57.226 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:57.263 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:57.333 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:57.333 INFO:tasks.ceph:role: osd.5 2023-11-28T11:07:57.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:07:57.333 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:07:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:57.409 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:57.409 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:57.824 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2023-11-28T11:07:57.826 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2023-11-28T11:07:57.826 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:57.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:57.971 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:58.041 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:58.041 INFO:tasks.ceph:role: osd.6 2023-11-28T11:07:58.041 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:07:58.041 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:07:58.116 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:58.116 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:58.116 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:58.117 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:58.565 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2023-11-28T11:07:58.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2023-11-28T11:07:58.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:58.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:58.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:58.836 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:58.836 INFO:tasks.ceph:role: osd.7 2023-11-28T11:07:58.836 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:07:58.837 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:58.913 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:58.914 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:58.914 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:58.914 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:59.356 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2023-11-28T11:07:59.358 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2023-11-28T11:07:59.359 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:59.531 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:59.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:59.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:07:59.673+0000 7f7ec5e2b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T11:07:59.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:07:59.674+0000 7f7ec5e2b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T11:07:59.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:07:59.674+0000 7f7ec5e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:07:59.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:07:59.674+0000 7f7ec5e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:07:59.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:07:59.674+0000 7f7ec5e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:07:59.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:07:59.674+0000 7f7ec5e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T11:08:02.555 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T11:08:02.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:02.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:02.672+0000 7f8503736640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T11:08:02.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:02.672+0000 7f8503736640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T11:08:02.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:02.672+0000 7f8503736640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:08:02.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:02.672+0000 7f8503736640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:08:02.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:02.672+0000 7f8503736640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:08:02.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:02.672+0000 7f8503736640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T11:08:05.556 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T11:08:08.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:08.924 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:08.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:09.043 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:09.041+0000 7f44a91aa640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T11:08:09.043 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:09.041+0000 7f44a91aa640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T11:08:09.043 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:09.041+0000 7f44a91aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:08:09.043 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:09.041+0000 7f44a91aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:08:09.043 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:09.041+0000 7f44a91aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:08:09.043 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:09.041+0000 7f44a91aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T11:08:11.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T11:08:11.990 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:12.080 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:12.078+0000 7f137e1b7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T11:08:12.081 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:12.079+0000 7f137e1b7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T11:08:12.081 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:12.079+0000 7f137e1b7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:08:12.081 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:12.079+0000 7f137e1b7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:08:12.081 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:12.079+0000 7f137e1b7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:08:12.081 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:12.079+0000 7f137e1b7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T11:08:14.939 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T11:08:14.970 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T11:08:14.971 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:14.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T11:08:15.003 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T11:08:15.074 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T11:08:15.147 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T11:08:15.218 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T11:08:15.288 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T11:08:15.356 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T11:08:15.424 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T11:08:15.456 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.456 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T11:08:15.527 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.527 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T11:08:15.596 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.596 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T11:08:15.665 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.665 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T11:08:15.734 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T11:08:15.802 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T11:08:15.870 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.870 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T11:08:15.891 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:15.891 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T11:08:15.951 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.951 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T11:08:15.968 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:15.968 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T11:08:16.025 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T11:08:16.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:08:16.028 DEBUG:teuthology.orchestra.run.smithi090:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:08:16.070 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2023-11-28T11:08:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD8yWVlHSGFKhAA3pr4QAikKKmqCfS0orTyWw== 2023-11-28T11:08:16.070 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2023-11-28T11:08:16.070 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD8yWVly5GSMRAArlawRsTlzQ8deoXsPnYmsg== 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD8yWVlCEbQOBAA8FKGXdCurT68yNjSUEJigQ== 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAAymVlMHf8JRAAqxuxtuaslP93VXhK17KfXA== 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQADymVlUrMkJRAAxCUtzDNzfHRTJ4jLzeTa0Q== 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAGymVlZqxOJBAAzhcB9ERpDVy5FCZNn1uMLg== 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAJymVlPZEhJBAAm8fhc9fRDDq3ZROV4Mez1Q== 2023-11-28T11:08:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD8yWVlBfMxLhAA6N4Cnno8jNhgzS5eYJKWqQ== 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD9yWVl2bkZAhAA15xpS3O6vPGdnCqW98I/0w== 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD9yWVlQFxUCRAA9BmlOX7cM5ewNextI+L6lg== 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAPymVlqwREKBAAsRlS/3ztcuWSZ9nRXZDKuA== 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQASymVlQKsoKBAABR2d0tZ/TLRKIDyQrAKVkA== 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAZymVlqduLAhAAPau0X+UZigxTAoWpmUnzRg== 2023-11-28T11:08:16.072 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAcymVlckzLBBAAH4vnm0MWED/F40ZWq2uIog== 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD9yWVllJ2YDRAAE9HeK1zyWWnOIRh09fAG3Q== 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout:[client.2] 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD9yWVlXJRCExAATH4e/FL5+nvYhdsZ7mB9LQ== 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD9yWVln8eKFhAAPGbMQ2n6PsaH6hEURGRJvw== 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout:[client.3] 2023-11-28T11:08:16.073 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD9yWVlo7c6HBAAHpELpEY29uzkqcJ+jAaxRA== 2023-11-28T11:08:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.y] 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD8yWVlHSGFKhAA3pr4QAikKKmqCfS0orTyWw== 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a] 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD8yWVly5GSMRAArlawRsTlzQ8deoXsPnYmsg== 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c] 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD8yWVlCEbQOBAA8FKGXdCurT68yNjSUEJigQ== 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.0] 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAAymVlMHf8JRAAqxuxtuaslP93VXhK17KfXA== 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.1] 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQADymVlUrMkJRAAxCUtzDNzfHRTJ4jLzeTa0Q== 2023-11-28T11:08:16.091 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.2] 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAGymVlZqxOJBAAzhcB9ERpDVy5FCZNn1uMLg== 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.3] 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAJymVlPZEhJBAAm8fhc9fRDDq3ZROV4Mez1Q== 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.x] 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD8yWVlBfMxLhAA6N4Cnno8jNhgzS5eYJKWqQ== 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b] 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD9yWVl2bkZAhAA15xpS3O6vPGdnCqW98I/0w== 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d] 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD9yWVlQFxUCRAA9BmlOX7cM5ewNextI+L6lg== 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.4] 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAPymVlqwREKBAAsRlS/3ztcuWSZ9nRXZDKuA== 2023-11-28T11:08:16.092 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.5] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQASymVlQKsoKBAABR2d0tZ/TLRKIDyQrAKVkA== 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.6] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAZymVlqduLAhAAPau0X+UZigxTAoWpmUnzRg== 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.7] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAcymVlckzLBBAAH4vnm0MWED/F40ZWq2uIog== 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD9yWVllJ2YDRAAE9HeK1zyWWnOIRh09fAG3Q== 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout:[client.2] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD9yWVlXJRCExAATH4e/FL5+nvYhdsZ7mB9LQ== 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout:[client.1] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD9yWVln8eKFhAAPGbMQ2n6PsaH6hEURGRJvw== 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout:[client.3] 2023-11-28T11:08:16.093 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQD9yWVlo7c6HBAAHpELpEY29uzkqcJ+jAaxRA== 2023-11-28T11:08:16.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:16.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:16.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.189 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.248 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.306 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.414 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:16.523 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:16.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.578 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.636 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:16.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.696 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.861 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:16.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:16.916 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:16.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:16.973 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:17.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:17.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:17.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:17.089 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:17.148 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T11:08:17.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T11:08:17.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:08:17.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T11:08:17.300 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T11:08:17.326 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:08:17.425 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T11:08:17.452 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T11:08:17.519 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:08:17.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T11:08:17.644 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:17.647 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:17.701 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T11:08:17.701 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T11:08:17.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T11:08:17.704 INFO:tasks.ceph.mon.a:Started 2023-11-28T11:08:17.704 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T11:08:17.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T11:08:17.744 INFO:tasks.ceph.mon.b:Started 2023-11-28T11:08:17.744 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T11:08:17.745 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T11:08:17.746 INFO:tasks.ceph.mon.c:Started 2023-11-28T11:08:17.746 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T11:08:17.747 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T11:08:17.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T11:08:17.749 INFO:tasks.ceph.mgr.y:Started 2023-11-28T11:08:17.749 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T11:08:17.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T11:08:17.751 INFO:tasks.ceph.mgr.x:Started 2023-11-28T11:08:17.751 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T11:08:17.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T11:08:17.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.875+0000 7f9c95624700 1 Processor -- start 2023-11-28T11:08:17.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.875+0000 7f9c95624700 1 -- start start 2023-11-28T11:08:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.875+0000 7f9c95624700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c900aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c95624700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 0x7f9c9006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c95624700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9006d4a0 con 0x7f9c9006e1a0 2023-11-28T11:08:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c95624700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9006e620 con 0x7f9c900b4bb0 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c95624700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b5020 con 0x7f9c9015c890 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9c9006e1a0 0x7f9c901604e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45390/0 (socket says 172.21.15.83:45390) 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8f7fe700 1 -- 172.21.15.83:0/60730243 learned_addr learned my addr 172.21.15.83:0/60730243 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8e7fc700 1 --2- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 0x7f9c9006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c900aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8e7fc700 1 -- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 msgr2=0x7f9c9006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:08:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8effd700 1 -- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 msgr2=0x7f9c900aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:08:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8e7fc700 1 -- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 msgr2=0x7f9c9006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:08:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8effd700 1 -- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 msgr2=0x7f9c900aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:08:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8e7fc700 1 --2- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 0x7f9c9006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:08:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c900aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:08:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c900aee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:08:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:17.876+0000 7f9c8e7fc700 1 --2- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 0x7f9c9006dc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:08:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.074+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735206137 0 0) 0x7f9c9006d4a0 con 0x7f9c9006e1a0 2023-11-28T11:08:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.074+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c780035b0 con 0x7f9c9006e1a0 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96656202 0 0) 0x7f9c780035b0 con 0x7f9c9006e1a0 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 msgr2=0x7f9c900aee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c900aee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 msgr2=0x7f9c9006dc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 --2- 172.21.15.83:0/60730243 >> v2:172.21.15.90:3300/0 conn(0x7f9c900b4bb0 0x7f9c9006dc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9006c160 con 0x7f9c9006e1a0 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f9c880019f0 con 0x7f9c9006e1a0 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 >> v1:172.21.15.83:6789/0 conn(0x7f9c9006e1a0 legacy=0x7f9c901604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c78003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c78004030 0x7f9c78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c78004d90 0x7f9c78005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9006e620 con 0x7f9c78004d90 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b5020 con 0x7f9c78004030 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7800bba0 con 0x7f9c9015c890 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.075+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c78003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8e7fc700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c78004030 0x7f9c78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8f7fe700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c78004d90 0x7f9c78005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8effd700 1 -- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c78004030 msgr2=0x7f9c78004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c78004030 0x7f9c78004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8effd700 1 -- 172.21.15.83:0/60730243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c78004d90 msgr2=0x7f9c78005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c78004d90 0x7f9c78005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8effd700 1 -- 172.21.15.83:0/60730243 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780035b0 con 0x7f9c9015c890 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8e7fc700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c78004030 0x7f9c78004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8effd700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c78003a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000ed0 tx=0x7f9c78005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ae6035eba6db3b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80005c20 con 0x7f9c9015c890 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80005da0 con 0x7f9c9015c890 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.076+0000 7f9c8dffb700 1 -- 172.21.15.83:0/60730243 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80004270 con 0x7f9c9015c890 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c95624700 1 -- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 msgr2=0x7f9c78003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c95624700 1 --2- 172.21.15.83:0/60730243 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c9015c890 0x7f9c78003a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000ed0 tx=0x7f9c78005880 comp rx=0 tx=0).stop 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c8f7fe700 1 -- 172.21.15.83:0/60730243 reap_dead start 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c95624700 1 -- 172.21.15.83:0/60730243 shutdown_connections 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c95624700 1 -- 172.21.15.83:0/60730243 >> 172.21.15.83:0/60730243 conn(0x7f9c900b2b60 msgr2=0x7f9c900b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c95624700 1 -- 172.21.15.83:0/60730243 shutdown_connections 2023-11-28T11:08:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.077+0000 7f9c95624700 1 -- 172.21.15.83:0/60730243 wait complete. 2023-11-28T11:08:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.078+0000 7f9c95624700 1 Processor -- start 2023-11-28T11:08:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.078+0000 7f9c95624700 1 -- start start 2023-11-28T11:08:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c95624700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9015c890 0x7f9c900ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c95624700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c900cbd70 0x7f9c901f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c95624700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 0x7f9c901f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c95624700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900cb560 con 0x7f9c9015c890 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c95624700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9015cf90 con 0x7f9c901f6d10 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c95624700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901f76f0 con 0x7f9c900cbd70 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c8f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 0x7f9c901f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c8e7fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c900cbd70 0x7f9c901f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c8f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 0x7f9c901f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:34550/0 (socket says 172.21.15.83:34550) 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c8e7fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c900cbd70 0x7f9c901f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:59988/0 (socket says 172.21.15.83:59988) 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.079+0000 7f9c8f7fe700 1 -- 172.21.15.83:0/2419448586 learned_addr learned my addr 172.21.15.83:0/2419448586 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c8f7fe700 1 -- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c900cbd70 msgr2=0x7f9c901f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c8f7fe700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c900cbd70 0x7f9c901f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c8f7fe700 1 -- 172.21.15.83:0/2419448586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9015c890 msgr2=0x7f9c900ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c8f7fe700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9015c890 0x7f9c900ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c8f7fe700 1 -- 172.21.15.83:0/2419448586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9006c400 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c8f7fe700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 0x7f9c901f71b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c900b2800 tx=0x7f9c8800a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800ad60 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c880133f0 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880137a0 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c900b5ae0 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.080+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c900b5240 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.081+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9c88023070 con 0x7f9c901f6d10 2023-11-28T11:08:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.081+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9c8801b740 con 0x7f9c901f6d10 2023-11-28T11:08:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.081+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c900cb6a0 con 0x7f9c901f6d10 2023-11-28T11:08:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.083+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9c900cb6a0 con 0x7f9c901f6d10 2023-11-28T11:08:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.159+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f9c900ca690 con 0x7f9c901f6d10 2023-11-28T11:08:18.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.167+0000 7f9c777fe700 1 -- 172.21.15.83:0/2419448586 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9c8802c090 con 0x7f9c901f6d10 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.167+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 msgr2=0x7f9c901f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 0x7f9c901f71b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c900b2800 tx=0x7f9c8800a5f0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 shutdown_connections 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c901f6d10 0x7f9c901f71b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9015c890 0x7f9c900ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 --2- 172.21.15.83:0/2419448586 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c900cbd70 0x7f9c901f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 >> 172.21.15.83:0/2419448586 conn(0x7f9c900b2b60 msgr2=0x7f9c900cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 shutdown_connections 2023-11-28T11:08:18.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:18.168+0000 7f9c95624700 1 -- 172.21.15.83:0/2419448586 wait complete. 2023-11-28T11:08:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2023-11-28T11:08:18.181 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T11:08:18.181 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T11:08:18.181 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:18.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T11:08:18.196 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:18.194+0000 7fa57ec07200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:08:18.202 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:18.200+0000 7f9fe51c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:08:18.207 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:18.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T11:08:18.263 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:18.262+0000 7fa57ec07200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:18.265 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:18.264+0000 7f9fe51c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:18.273 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:18.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T11:08:18.339 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:18.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T11:08:18.340 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:18.339+0000 7f9fe51c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:08:18.342 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:18.341+0000 7fa57ec07200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:08:18.367 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:18.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T11:08:18.394 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:18.394 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T11:08:18.421 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:18.419+0000 7f9fe51c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:08:18.425 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:18.424+0000 7fa57ec07200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:08:18.446 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:18.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T11:08:18.512 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:08:18.512 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T11:08:18.566 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:18.565+0000 7fa57ec07200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:08:18.569 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:18.568+0000 7f9fe51c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:08:18.578 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new b3e54911-ebed-4e23-9277-fae35bddfa94 0 2023-11-28T11:08:18.741 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.737+0000 7f3e52533700 1 Processor -- start 2023-11-28T11:08:18.741 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.737+0000 7f3e52533700 1 -- start start 2023-11-28T11:08:18.741 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.737+0000 7f3e52533700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 0x7f3e4c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.741 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.737+0000 7f3e52533700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e4c0b4bb0 0x7f3e4c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.741 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.737+0000 7f3e52533700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3e4c06e1a0 0x7f3e4c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e52533700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c06e620 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e52533700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c0b5020 con 0x7f3e4c06e1a0 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e52533700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c06d4a0 con 0x7f3e4c15c890 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 0x7f3e4c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e50ad0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3e4c06e1a0 0x7f3e4c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 0x7f3e4c0aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44090/0 (socket says 172.21.15.90:44090) 2023-11-28T11:08:18.742 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 -- 172.21.15.90:0/1341208226 learned_addr learned my addr 172.21.15.90:0/1341208226 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:18.743 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4b7fe700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.83:3300/0 conn(0x7f3e4c0b4bb0 0x7f3e4c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.743 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 -- 172.21.15.90:0/1341208226 >> v2:172.21.15.90:3300/0 conn(0x7f3e4c06e1a0 msgr2=0x7f3e4c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:18.743 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.90:3300/0 conn(0x7f3e4c06e1a0 0x7f3e4c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.743 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 -- 172.21.15.90:0/1341208226 >> v2:172.21.15.83:3300/0 conn(0x7f3e4c0b4bb0 msgr2=0x7f3e4c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:18.743 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.83:3300/0 conn(0x7f3e4c0b4bb0 0x7f3e4c06dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 -- 172.21.15.90:0/1341208226 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0b5ae0 con 0x7f3e4c15c890 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4b7fe700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.83:3300/0 conn(0x7f3e4c0b4bb0 0x7f3e4c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e50ad0700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.90:3300/0 conn(0x7f3e4c06e1a0 0x7f3e4c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4bfff700 1 --2- 172.21.15.90:0/1341208226 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 0x7f3e4c0aee60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c006fd0 tx=0x7f3e3c009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1142b4bf1fc64f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4affd700 1 -- 172.21.15.90:0/1341208226 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c005e30 con 0x7f3e4c15c890 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4affd700 1 -- 172.21.15.90:0/1341208226 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c004030 con 0x7f3e4c15c890 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.738+0000 7f3e4affd700 1 -- 172.21.15.90:0/1341208226 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0043e0 con 0x7f3e4c15c890 2023-11-28T11:08:18.744 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 -- 172.21.15.90:0/1341208226 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 msgr2=0x7f3e4c0aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 --2- 172.21.15.90:0/1341208226 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 0x7f3e4c0aee60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c006fd0 tx=0x7f3e3c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 -- 172.21.15.90:0/1341208226 shutdown_connections 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 --2- 172.21.15.90:0/1341208226 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c15c890 0x7f3e4c0aee60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.90:3300/0 conn(0x7f3e4c06e1a0 0x7f3e4c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 --2- 172.21.15.90:0/1341208226 >> v2:172.21.15.83:3300/0 conn(0x7f3e4c0b4bb0 0x7f3e4c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 -- 172.21.15.90:0/1341208226 >> 172.21.15.90:0/1341208226 conn(0x7f3e4c0b2b60 msgr2=0x7f3e4c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 -- 172.21.15.90:0/1341208226 shutdown_connections 2023-11-28T11:08:18.745 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.739+0000 7f3e52533700 1 -- 172.21.15.90:0/1341208226 wait complete. 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.740+0000 7f3e52533700 1 Processor -- start 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.740+0000 7f3e52533700 1 -- start start 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e52533700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 0x7f3e4c09d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e52533700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c0b4bb0 0x7f3e4c09a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e52533700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e4c15c890 0x7f3e4c14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e52533700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0c57d0 con 0x7f3e4c15c890 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e52533700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0c50e0 con 0x7f3e4c06e1a0 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e52533700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c09ac10 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 0x7f3e4c09d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.748 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4b7fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c0b4bb0 0x7f3e4c09a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 0x7f3e4c09d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48994/0 (socket says 172.21.15.90:48994) 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4bfff700 1 -- 172.21.15.90:0/962333994 learned_addr learned my addr 172.21.15.90:0/962333994 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e50ad0700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e4c15c890 0x7f3e4c14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4b7fe700 1 -- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 msgr2=0x7f3e4c09d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4b7fe700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 0x7f3e4c09d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4b7fe700 1 -- 172.21.15.90:0/962333994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e4c15c890 msgr2=0x7f3e4c14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4b7fe700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e4c15c890 0x7f3e4c14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4b7fe700 1 -- 172.21.15.90:0/962333994 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44009a70 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.741+0000 7f3e4bfff700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 0x7f3e4c09d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e4b7fe700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c0b4bb0 0x7f3e4c09a6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000e600 tx=0x7f3e4000e910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004230 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.749 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40004390 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c009500 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400036a0 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c0b5240 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3e40022070 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.742+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e40013d60 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.743+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c06c400 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.750 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.745+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3e40020060 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.826+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b3e54911-ebed-4e23-9277-fae35bddfa94", "id": 0} v 0) v1 -- 0x7f3e4c0d0bb0 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.833 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.831+0000 7f3e48ff9700 1 -- 172.21.15.90:0/962333994 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3e54911-ebed-4e23-9277-fae35bddfa94", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3e4000f040 con 0x7f3e4c0b4bb0 2023-11-28T11:08:18.833 INFO:teuthology.orchestra.run.smithi090.stdout:0 2023-11-28T11:08:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c0b4bb0 msgr2=0x7f3e4c09a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c0b4bb0 0x7f3e4c09a6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000e600 tx=0x7f3e4000e910 comp rx=0 tx=0).stop 2023-11-28T11:08:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 shutdown_connections 2023-11-28T11:08:18.835 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e4c15c890 0x7f3e4c14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.835 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c06e1a0 0x7f3e4c09d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.835 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 --2- 172.21.15.90:0/962333994 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e4c0b4bb0 0x7f3e4c09a6d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.836 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 >> 172.21.15.90:0/962333994 conn(0x7f3e4c0b2b60 msgr2=0x7f3e4c0ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.836 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 shutdown_connections 2023-11-28T11:08:18.836 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.832+0000 7f3e52533700 1 -- 172.21.15.90:0/962333994 wait complete. 2023-11-28T11:08:18.846 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new b3ea33d8-2165-4137-bf95-4e525e5171e8 1 2023-11-28T11:08:18.966 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.964+0000 7f91975f2700 1 Processor -- start 2023-11-28T11:08:18.966 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.964+0000 7f91975f2700 1 -- start start 2023-11-28T11:08:18.969 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f91975f2700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 0x7f91900aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.969 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f91975f2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f919006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.969 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f91975f2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919006d4a0 con 0x7f919006e1a0 2023-11-28T11:08:18.969 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f91975f2700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919006e620 con 0x7f919015c890 2023-11-28T11:08:18.969 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f91975f2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91900b5020 con 0x7f91900b4bb0 2023-11-28T11:08:18.969 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f919006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f919538e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 0x7f91900aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f919538e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 0x7f91900aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49000/0 (socket says 172.21.15.90:49000) 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f919006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44104/0 (socket says 172.21.15.90:44104) 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f919538e700 1 -- 172.21.15.90:0/1692737538 learned_addr learned my addr 172.21.15.90:0/1692737538 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 -- 172.21.15.90:0/1692737538 >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 msgr2=0x7f91900aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 --2- 172.21.15.90:0/1692737538 >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 0x7f91900aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 -- 172.21.15.90:0/1692737538 >> v1:172.21.15.83:6789/0 conn(0x7f919006e1a0 legacy=0x7f91901604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 -- 172.21.15.90:0/1692737538 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919006c160 con 0x7f91900b4bb0 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f919538e700 1 --2- 172.21.15.90:0/1692737538 >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 0x7f91900aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.965+0000 7f9194b8d700 1 --2- 172.21.15.90:0/1692737538 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f919006dc60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9180009870 tx=0x7f9180009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2549dbcb934e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.970 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f9187fff700 1 -- 172.21.15.90:0/1692737538 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011830 con 0x7f91900b4bb0 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f9187fff700 1 -- 172.21.15.90:0/1692737538 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180011990 con 0x7f91900b4bb0 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f9187fff700 1 -- 172.21.15.90:0/1692737538 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011d20 con 0x7f91900b4bb0 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 -- 172.21.15.90:0/1692737538 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 msgr2=0x7f919006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 --2- 172.21.15.90:0/1692737538 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f919006dc60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9180009870 tx=0x7f9180009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 -- 172.21.15.90:0/1692737538 shutdown_connections 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 --2- 172.21.15.90:0/1692737538 >> v2:172.21.15.90:3300/0 conn(0x7f919015c890 0x7f91900aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 --2- 172.21.15.90:0/1692737538 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f919006dc60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 -- 172.21.15.90:0/1692737538 >> 172.21.15.90:0/1692737538 conn(0x7f91900b2b60 msgr2=0x7f91900b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 -- 172.21.15.90:0/1692737538 shutdown_connections 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.966+0000 7f91975f2700 1 -- 172.21.15.90:0/1692737538 wait complete. 2023-11-28T11:08:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 Processor -- start 2023-11-28T11:08:18.972 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 -- start start 2023-11-28T11:08:18.974 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 0x7f9190158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.974 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f91900c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.975 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f919015c890 0x7f9190149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.975 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91900b2650 con 0x7f919015c890 2023-11-28T11:08:18.975 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91900afbf0 con 0x7f919006e1a0 2023-11-28T11:08:18.975 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f91975f2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91900ca460 con 0x7f91900b4bb0 2023-11-28T11:08:18.975 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f919538e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 0x7f9190158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.976 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f9194b8d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f91900c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.976 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f919538e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 0x7f9190158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49008/0 (socket says 172.21.15.90:49008) 2023-11-28T11:08:18.976 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f919538e700 1 -- 172.21.15.90:0/3849639877 learned_addr learned my addr 172.21.15.90:0/3849639877 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:18.976 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.968+0000 7f9195b8f700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f919015c890 0x7f9190149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.976 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f919538e700 1 -- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 msgr2=0x7f91900c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.976 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f919538e700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f91900c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.977 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f919538e700 1 -- 172.21.15.90:0/3849639877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f919015c890 msgr2=0x7f9190149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.977 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f919538e700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f919015c890 0x7f9190149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.977 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f919538e700 1 -- 172.21.15.90:0/3849639877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c009500 con 0x7f919006e1a0 2023-11-28T11:08:18.977 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f9194b8d700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f91900c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f919538e700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 0x7f9190158620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9a0 tx=0x7f918c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c005a20 con 0x7f919006e1a0 2023-11-28T11:08:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c005b80 con 0x7f919006e1a0 2023-11-28T11:08:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c00c690 con 0x7f919006e1a0 2023-11-28T11:08:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91800094a0 con 0x7f919006e1a0 2023-11-28T11:08:18.979 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f918c019070 con 0x7f919006e1a0 2023-11-28T11:08:18.979 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919006c400 con 0x7f919006e1a0 2023-11-28T11:08:18.979 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.969+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f918c006840 con 0x7f919006e1a0 2023-11-28T11:08:18.979 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.970+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91900b5ae0 con 0x7f919006e1a0 2023-11-28T11:08:18.979 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:18.972+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f918c03ab00 con 0x7f919006e1a0 2023-11-28T11:08:19.056 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.054+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b3ea33d8-2165-4137-bf95-4e525e5171e8", "id": 1} v 0) v1 -- 0x7f91900b5240 con 0x7f919006e1a0 2023-11-28T11:08:19.062 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.060+0000 7f9185ffb700 1 -- 172.21.15.90:0/3849639877 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3ea33d8-2165-4137-bf95-4e525e5171e8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f918c029030 con 0x7f919006e1a0 2023-11-28T11:08:19.062 INFO:teuthology.orchestra.run.smithi090.stdout:1 2023-11-28T11:08:19.063 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.061+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 msgr2=0x7f9190158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.063 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.061+0000 7f91975f2700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 0x7f9190158620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9a0 tx=0x7f918c00ecb0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.063 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 shutdown_connections 2023-11-28T11:08:19.064 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f919015c890 0x7f9190149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.064 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f919006e1a0 0x7f9190158620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.064 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 --2- 172.21.15.90:0/3849639877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91900b4bb0 0x7f91900c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.064 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 >> 172.21.15.90:0/3849639877 conn(0x7f91900b2b60 msgr2=0x7f91900adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.064 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 shutdown_connections 2023-11-28T11:08:19.064 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.062+0000 7f91975f2700 1 -- 172.21.15.90:0/3849639877 wait complete. 2023-11-28T11:08:19.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 30f1202f-b122-4cf7-a737-7d71678bee11 2 2023-11-28T11:08:19.195 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.193+0000 7f40d953c700 1 Processor -- start 2023-11-28T11:08:19.196 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.194+0000 7f40d953c700 1 -- start start 2023-11-28T11:08:19.202 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.194+0000 7f40d953c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d406e370 0x7f40d40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.202 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.194+0000 7f40d953c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d406d4a0 con 0x7f40d40b4bb0 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.194+0000 7f40d953c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40b5ae0 con 0x7f40d415c890 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.194+0000 7f40d953c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40b5050 con 0x7f40d406e370 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d27fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d406e370 0x7f40d40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d2ffd700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f40d415c890 0x7f40d406dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:55248/0 (socket says 172.21.15.90:55248) 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d2ffd700 1 -- 172.21.15.90:0/2259217919 learned_addr learned my addr 172.21.15.90:0/2259217919 (peer_addr_for_me v1:172.21.15.90:0/0) 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d1ffb700 1 -- 172.21.15.90:0/2259217919 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408390214 0 0) 0x7f40d40b5ae0 con 0x7f40d415c890 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d27fc700 1 -- 172.21.15.90:0/2259217919 >> v1:172.21.15.90:6789/0 conn(0x7f40d415c890 legacy=0x7f40d406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d27fc700 1 -- 172.21.15.90:0/2259217919 >> v1:172.21.15.83:6789/0 conn(0x7f40d40b4bb0 legacy=0x7f40d406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d27fc700 1 -- 172.21.15.90:0/2259217919 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d406c400 con 0x7f40d406e370 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d27fc700 1 --2- 172.21.15.90:0/2259217919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d406e370 0x7f40d40aee60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009870 tx=0x7f40c8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42796a36d873e209 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d1ffb700 1 -- 172.21.15.90:0/2259217919 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011830 con 0x7f40d406e370 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d1ffb700 1 -- 172.21.15.90:0/2259217919 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8011990 con 0x7f40d406e370 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.195+0000 7f40d1ffb700 1 -- 172.21.15.90:0/2259217919 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011d20 con 0x7f40d406e370 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 -- 172.21.15.90:0/2259217919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d406e370 msgr2=0x7f40d40aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 --2- 172.21.15.90:0/2259217919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d406e370 0x7f40d40aee60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009870 tx=0x7f40c8009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 -- 172.21.15.90:0/2259217919 shutdown_connections 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 --2- 172.21.15.90:0/2259217919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d406e370 0x7f40d40aee60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 -- 172.21.15.90:0/2259217919 >> 172.21.15.90:0/2259217919 conn(0x7f40d40b2b60 msgr2=0x7f40d40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.204 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 -- 172.21.15.90:0/2259217919 shutdown_connections 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.196+0000 7f40d953c700 1 -- 172.21.15.90:0/2259217919 wait complete. 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.197+0000 7f40d953c700 1 Processor -- start 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.197+0000 7f40d953c700 1 -- start start 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d406e370 0x7f40d415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 0x7f40d40c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40d415c890 0x7f40d414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415b0e0 con 0x7f40d406e370 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415cc60 con 0x7f40d415c890 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40c3c80 con 0x7f40d40b4bb0 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 0x7f40d40c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 0x7f40d40c3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44122/0 (socket says 172.21.15.90:44122) 2023-11-28T11:08:19.205 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 -- 172.21.15.90:0/2182515488 learned_addr learned my addr 172.21.15.90:0/2182515488 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d37fe700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40d415c890 0x7f40d414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d2ffd700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d406e370 0x7f40d415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 -- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40d415c890 msgr2=0x7f40d414a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40d415c890 0x7f40d414a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 -- 172.21.15.90:0/2182515488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d406e370 msgr2=0x7f40d415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d406e370 0x7f40d415f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 -- 172.21.15.90:0/2182515488 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80094a0 con 0x7f40d40b4bb0 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d27fc700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 0x7f40d40c3740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40c80090b0 tx=0x7f40c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80036a0 con 0x7f40d40b4bb0 2023-11-28T11:08:19.206 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c801e070 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d40b5520 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.198+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8003a60 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.199+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d40b5240 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.199+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f40c800f050 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.199+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f40c800b840 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.199+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d40b5800 con 0x7f40d40b4bb0 2023-11-28T11:08:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.203+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f40c8020070 con 0x7f40d40b4bb0 2023-11-28T11:08:19.252 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:19.251+0000 7fa57ec07200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:08:19.288 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.286+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd new", "uuid": "30f1202f-b122-4cf7-a737-7d71678bee11", "id": 2} v 0) v1 -- 0x7f40d415cda0 con 0x7f40d40b4bb0 2023-11-28T11:08:19.296 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.294+0000 7f40bb7fe700 1 -- 172.21.15.90:0/2182515488 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "30f1202f-b122-4cf7-a737-7d71678bee11", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f40c80320b0 con 0x7f40d40b4bb0 2023-11-28T11:08:19.296 INFO:teuthology.orchestra.run.smithi090.stdout:2 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 msgr2=0x7f40d40c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 0x7f40d40c3740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40c80090b0 tx=0x7f40c8004750 comp rx=0 tx=0).stop 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 shutdown_connections 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40d415c890 0x7f40d414a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d406e370 0x7f40d415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 --2- 172.21.15.90:0/2182515488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40d40b4bb0 0x7f40d40c3740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.297 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 >> 172.21.15.90:0/2182515488 conn(0x7f40d40b2b60 msgr2=0x7f40d40ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.298 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 shutdown_connections 2023-11-28T11:08:19.298 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.295+0000 7f40d953c700 1 -- 172.21.15.90:0/2182515488 wait complete. 2023-11-28T11:08:19.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 1f803565-3bcc-4627-8bd4-157d7a1be074 3 2023-11-28T11:08:19.309 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:19.307+0000 7f9fe51c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:08:19.327 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:19.326+0000 7fa57ec07200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:08:19.389 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:19.387+0000 7f9fe51c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:08:19.427 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:19.426+0000 7fa57ec07200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:08:19.428 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.427+0000 7f1e45aae700 1 Processor -- start 2023-11-28T11:08:19.429 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.427+0000 7f1e45aae700 1 -- start start 2023-11-28T11:08:19.430 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e45aae700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1e40158bc0 0x7f1e400cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.430 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e45aae700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 0x7f1e400ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.430 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e45aae700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e40158f90 con 0x7f1e400cb680 2023-11-28T11:08:19.430 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e45aae700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4015c840 con 0x7f1e40158bc0 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e45aae700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015cb20 con 0x7f1e40071a50 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1e40158bc0 0x7f1e400cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 0x7f1e400ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 0x7f1e400ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44130/0 (socket says 172.21.15.90:44130) 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1e40158bc0 0x7f1e400cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49024/0 (socket says 172.21.15.90:49024) 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3ffff700 1 -- 172.21.15.90:0/2289675684 learned_addr learned my addr 172.21.15.90:0/2289675684 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 -- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 msgr2=0x7f1e400ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:19.431 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 --2- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 0x7f1e400ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 -- 172.21.15.90:0/2289675684 >> v1:172.21.15.90:6789/0 conn(0x7f1e400cb680 legacy=0x7f1e40071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 -- 172.21.15.90:0/2289675684 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4015ce00 con 0x7f1e40158bc0 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3ffff700 1 --2- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 0x7f1e400ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3f7fe700 1 --2- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e40158bc0 0x7f1e400cdd70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009840 tx=0x7f1e30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d520482d22ef63 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.428+0000 7f1e3e7fc700 1 -- 172.21.15.90:0/2289675684 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000f8a0 con 0x7f1e40158bc0 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e3e7fc700 1 -- 172.21.15.90:0/2289675684 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000fa00 con 0x7f1e40158bc0 2023-11-28T11:08:19.432 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e3e7fc700 1 -- 172.21.15.90:0/2289675684 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000fd90 con 0x7f1e40158bc0 2023-11-28T11:08:19.436 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 -- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e40158bc0 msgr2=0x7f1e400cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.436 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 --2- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e40158bc0 0x7f1e400cdd70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009840 tx=0x7f1e30009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:19.436 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 -- 172.21.15.90:0/2289675684 shutdown_connections 2023-11-28T11:08:19.436 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 --2- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40071a50 0x7f1e400ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.436 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 --2- 172.21.15.90:0/2289675684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e40158bc0 0x7f1e400cdd70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.436 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 -- 172.21.15.90:0/2289675684 >> 172.21.15.90:0/2289675684 conn(0x7f1e400cf300 msgr2=0x7f1e400cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 -- 172.21.15.90:0/2289675684 shutdown_connections 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.429+0000 7f1e45aae700 1 -- 172.21.15.90:0/2289675684 wait complete. 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.431+0000 7f1e45aae700 1 Processor -- start 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.431+0000 7f1e45aae700 1 -- start start 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e45aae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e40071a50 0x7f1e40153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e45aae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e400cb680 0x7f1e401f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e45aae700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 0x7f1e4015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e45aae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015b8f0 con 0x7f1e40071a50 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e45aae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401540b0 con 0x7f1e400cb680 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e45aae700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015b330 con 0x7f1e40158bc0 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 0x7f1e4015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.437 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 0x7f1e4015aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44134/0 (socket says 172.21.15.90:44134) 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3ffff700 1 -- 172.21.15.90:0/853240684 learned_addr learned my addr 172.21.15.90:0/853240684 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e400cb680 0x7f1e401f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3f7fe700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e40071a50 0x7f1e40153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 -- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 msgr2=0x7f1e4015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 0x7f1e4015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 -- 172.21.15.90:0/853240684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e40071a50 msgr2=0x7f1e40153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e40071a50 0x7f1e40153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 -- 172.21.15.90:0/853240684 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3800ac40 con 0x7f1e400cb680 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3ffff700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 0x7f1e4015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e3effd700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e400cb680 0x7f1e401f31f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e34000f60 tx=0x7f1e3400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.432+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340052d0 con 0x7f1e400cb680 2023-11-28T11:08:19.438 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e34005430 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340038c0 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e300094a0 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4015d3c0 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f1e34019030 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1e34011bb0 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.433+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4015d0e0 con 0x7f1e400cb680 2023-11-28T11:08:19.439 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.437+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f1e34017070 con 0x7f1e400cb680 2023-11-28T11:08:19.470 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:19.468+0000 7f9fe51c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:08:19.522 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.520+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1f803565-3bcc-4627-8bd4-157d7a1be074", "id": 3} v 0) v1 -- 0x7f1e400ce2b0 con 0x7f1e400cb680 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.528+0000 7f1e27fff700 1 -- 172.21.15.90:0/853240684 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1f803565-3bcc-4627-8bd4-157d7a1be074", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1e34011e10 con 0x7f1e400cb680 2023-11-28T11:08:19.531 INFO:teuthology.orchestra.run.smithi090.stdout:3 2023-11-28T11:08:19.532 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e400cb680 msgr2=0x7f1e401f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.533 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e400cb680 0x7f1e401f31f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e34000f60 tx=0x7f1e3400c730 comp rx=0 tx=0).stop 2023-11-28T11:08:19.533 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 shutdown_connections 2023-11-28T11:08:19.533 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1e40158bc0 0x7f1e4015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.533 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e40071a50 0x7f1e40153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.533 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 --2- 172.21.15.90:0/853240684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e400cb680 0x7f1e401f31f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.534 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 >> 172.21.15.90:0/853240684 conn(0x7f1e400cf300 msgr2=0x7f1e400cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.534 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 shutdown_connections 2023-11-28T11:08:19.534 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.529+0000 7f1e45aae700 1 -- 172.21.15.90:0/853240684 wait complete. 2023-11-28T11:08:19.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 59c878b2-da83-47b2-a5fb-fdf4fefc14b5 4 2023-11-28T11:08:19.563 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:19.561+0000 7fa57ec07200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:08:19.612 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:19.610+0000 7f9fe51c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:08:19.664 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.662+0000 7f21ca76e700 1 Processor -- start 2023-11-28T11:08:19.664 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.662+0000 7f21ca76e700 1 -- start start 2023-11-28T11:08:19.670 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21ca76e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21c415c890 0x7f21c40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.670 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21ca76e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f21c40b4bb0 0x7f21c406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.670 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21ca76e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c406d4a0 con 0x7f21c406e1a0 2023-11-28T11:08:19.671 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21ca76e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c406e620 con 0x7f21c40b4bb0 2023-11-28T11:08:19.671 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21ca76e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40b5020 con 0x7f21c415c890 2023-11-28T11:08:19.671 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21c415c890 0x7f21c40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.671 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21c415c890 0x7f21c40aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49036/0 (socket says 172.21.15.90:49036) 2023-11-28T11:08:19.671 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 -- 172.21.15.90:0/4268589155 learned_addr learned my addr 172.21.15.90:0/4268589155 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:19.672 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c37fe700 1 --2- 172.21.15.90:0/4268589155 >> v2:172.21.15.83:3300/0 conn(0x7f21c40b4bb0 0x7f21c406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.672 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 -- 172.21.15.90:0/4268589155 >> v2:172.21.15.83:3300/0 conn(0x7f21c40b4bb0 msgr2=0x7f21c406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:19.672 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 --2- 172.21.15.90:0/4268589155 >> v2:172.21.15.83:3300/0 conn(0x7f21c40b4bb0 0x7f21c406dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.672 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 -- 172.21.15.90:0/4268589155 >> v1:172.21.15.83:6789/0 conn(0x7f21c406e1a0 legacy=0x7f21c41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.672 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 -- 172.21.15.90:0/4268589155 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c406c160 con 0x7f21c415c890 2023-11-28T11:08:19.672 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c3fff700 1 --2- 172.21.15.90:0/4268589155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c415c890 0x7f21c40aee60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21c40715b0 tx=0x7f21b4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac25be731366485 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.673 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c2ffd700 1 -- 172.21.15.90:0/4268589155 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4005e30 con 0x7f21c415c890 2023-11-28T11:08:19.673 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c2ffd700 1 -- 172.21.15.90:0/4268589155 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4004030 con 0x7f21c415c890 2023-11-28T11:08:19.673 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.663+0000 7f21c2ffd700 1 -- 172.21.15.90:0/4268589155 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40043e0 con 0x7f21c415c890 2023-11-28T11:08:19.673 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 -- 172.21.15.90:0/4268589155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c415c890 msgr2=0x7f21c40aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.673 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 --2- 172.21.15.90:0/4268589155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c415c890 0x7f21c40aee60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21c40715b0 tx=0x7f21b4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.673 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 -- 172.21.15.90:0/4268589155 shutdown_connections 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 --2- 172.21.15.90:0/4268589155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c415c890 0x7f21c40aee60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 --2- 172.21.15.90:0/4268589155 >> v2:172.21.15.83:3300/0 conn(0x7f21c40b4bb0 0x7f21c406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 -- 172.21.15.90:0/4268589155 >> 172.21.15.90:0/4268589155 conn(0x7f21c40b2b60 msgr2=0x7f21c40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 -- 172.21.15.90:0/4268589155 shutdown_connections 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.664+0000 7f21ca76e700 1 -- 172.21.15.90:0/4268589155 wait complete. 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.665+0000 7f21ca76e700 1 Processor -- start 2023-11-28T11:08:19.674 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.665+0000 7f21ca76e700 1 -- start start 2023-11-28T11:08:19.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21ca76e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c406e1a0 0x7f21c4157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21ca76e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 0x7f21c41f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21ca76e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f21c415c890 0x7f21c415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21ca76e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41590e0 con 0x7f21c406e1a0 2023-11-28T11:08:19.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21ca76e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41f32a0 con 0x7f21c40b4bb0 2023-11-28T11:08:19.675 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21ca76e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40c71e0 con 0x7f21c415c890 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21c37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 0x7f21c41f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21c37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 0x7f21c41f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49044/0 (socket says 172.21.15.90:49044) 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21c37fe700 1 -- 172.21.15.90:0/961900445 learned_addr learned my addr 172.21.15.90:0/961900445 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21c8d0b700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f21c415c890 0x7f21c415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.666+0000 7f21c3fff700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c406e1a0 0x7f21c4157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c37fe700 1 -- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f21c415c890 msgr2=0x7f21c415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.676 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c37fe700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f21c415c890 0x7f21c415aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c37fe700 1 -- 172.21.15.90:0/961900445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c406e1a0 msgr2=0x7f21c4157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c37fe700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c406e1a0 0x7f21c4157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c37fe700 1 -- 172.21.15.90:0/961900445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4009500 con 0x7f21c40b4bb0 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c37fe700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 0x7f21c41f2d60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d800 tx=0x7f21b800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80125f0 con 0x7f21c40b4bb0 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8012750 con 0x7f21c40b4bb0 2023-11-28T11:08:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c406c400 con 0x7f21c40b4bb0 2023-11-28T11:08:19.678 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8012b00 con 0x7f21c40b4bb0 2023-11-28T11:08:19.678 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c40b5ae0 con 0x7f21c40b4bb0 2023-11-28T11:08:19.678 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f21b8013480 con 0x7f21c40b4bb0 2023-11-28T11:08:19.678 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.667+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f21b801d970 con 0x7f21c40b4bb0 2023-11-28T11:08:19.678 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.668+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c40b5240 con 0x7f21c40b4bb0 2023-11-28T11:08:19.679 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.670+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f21b801d510 con 0x7f21c40b4bb0 2023-11-28T11:08:19.756 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.754+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "59c878b2-da83-47b2-a5fb-fdf4fefc14b5", "id": 4} v 0) v1 -- 0x7f21c40c80d0 con 0x7f21c40b4bb0 2023-11-28T11:08:19.763 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.762+0000 7f21c0ff9700 1 -- 172.21.15.90:0/961900445 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "59c878b2-da83-47b2-a5fb-fdf4fefc14b5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f21b8005a20 con 0x7f21c40b4bb0 2023-11-28T11:08:19.764 INFO:teuthology.orchestra.run.smithi090.stdout:4 2023-11-28T11:08:19.765 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 msgr2=0x7f21c41f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.765 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 0x7f21c41f2d60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d800 tx=0x7f21b800db10 comp rx=0 tx=0).stop 2023-11-28T11:08:19.765 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 shutdown_connections 2023-11-28T11:08:19.766 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f21c415c890 0x7f21c415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.766 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c406e1a0 0x7f21c4157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.766 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 --2- 172.21.15.90:0/961900445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c40b4bb0 0x7f21c41f2d60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.766 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 >> 172.21.15.90:0/961900445 conn(0x7f21c40b2b60 msgr2=0x7f21c40af670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.766 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 shutdown_connections 2023-11-28T11:08:19.766 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.763+0000 7f21ca76e700 1 -- 172.21.15.90:0/961900445 wait complete. 2023-11-28T11:08:19.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new d04ea464-897d-4c86-b799-f04692ae0f16 5 2023-11-28T11:08:19.900 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.899+0000 7f421ad46700 1 Processor -- start 2023-11-28T11:08:19.900 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.899+0000 7f421ad46700 1 -- start start 2023-11-28T11:08:19.903 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f421ad46700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f421406a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.903 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f421ad46700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42140cd520 con 0x7f42140b4940 2023-11-28T11:08:19.903 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f421ad46700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42140cf400 con 0x7f42140d2880 2023-11-28T11:08:19.903 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f421ad46700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42140cd7c0 con 0x7f42140b8570 2023-11-28T11:08:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f4213fff700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f42140d2880 0x7f4214160c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:55286/0 (socket says 172.21.15.90:55286) 2023-11-28T11:08:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f4218ae2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f421406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f4213fff700 1 -- 172.21.15.90:0/4218360527 learned_addr learned my addr 172.21.15.90:0/4218360527 (peer_addr_for_me v1:172.21.15.90:0/0) 2023-11-28T11:08:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f42137fe700 1 -- 172.21.15.90:0/4218360527 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691861945 0 0) 0x7f42140cf400 con 0x7f42140d2880 2023-11-28T11:08:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f42137fe700 1 -- 172.21.15.90:0/4218360527 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42040035b0 con 0x7f42140d2880 2023-11-28T11:08:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f4218ae2700 1 -- 172.21.15.90:0/4218360527 >> v1:172.21.15.90:6789/0 conn(0x7f42140d2880 legacy=0x7f4214160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f42137fe700 1 -- 172.21.15.90:0/4218360527 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387104995 0 0) 0x7f42140cd520 con 0x7f42140b4940 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f4218ae2700 1 -- 172.21.15.90:0/4218360527 >> v1:172.21.15.83:6789/0 conn(0x7f42140b4940 legacy=0x7f42140d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.900+0000 7f4218ae2700 1 -- 172.21.15.90:0/4218360527 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140cdaa0 con 0x7f42140b8570 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f4218ae2700 1 --2- 172.21.15.90:0/4218360527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f421406a270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4208009870 tx=0x7f4208009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c08ed0dabe30e75 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f42137fe700 1 -- 172.21.15.90:0/4218360527 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208011830 con 0x7f42140b8570 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f42137fe700 1 -- 172.21.15.90:0/4218360527 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4208011990 con 0x7f42140b8570 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f42137fe700 1 -- 172.21.15.90:0/4218360527 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208011d20 con 0x7f42140b8570 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 -- 172.21.15.90:0/4218360527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 msgr2=0x7f421406a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 --2- 172.21.15.90:0/4218360527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f421406a270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4208009870 tx=0x7f4208009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 -- 172.21.15.90:0/4218360527 shutdown_connections 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 --2- 172.21.15.90:0/4218360527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f421406a270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 -- 172.21.15.90:0/4218360527 >> 172.21.15.90:0/4218360527 conn(0x7f42140d30a0 msgr2=0x7f42140d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 -- 172.21.15.90:0/4218360527 shutdown_connections 2023-11-28T11:08:19.906 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.901+0000 7f421ad46700 1 -- 172.21.15.90:0/4218360527 wait complete. 2023-11-28T11:08:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 Processor -- start 2023-11-28T11:08:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 -- start start 2023-11-28T11:08:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 0x7f42140cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f42141489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42140d2880 0x7f4214148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214155130 con 0x7f42140d2880 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42141553d0 con 0x7f42140b4940 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.903+0000 7f421ad46700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42141556b0 con 0x7f42140b8570 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4213fff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f42141489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 0x7f42140cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4213fff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f42141489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44158/0 (socket says 172.21.15.90:44158) 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4213fff700 1 -- 172.21.15.90:0/2252624461 learned_addr learned my addr 172.21.15.90:0/2252624461 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 0x7f42140cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49054/0 (socket says 172.21.15.90:49054) 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f42192e3700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42140d2880 0x7f4214148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 -- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 msgr2=0x7f42141489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f42141489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 -- 172.21.15.90:0/2252624461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42140d2880 msgr2=0x7f4214148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.910 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42140d2880 0x7f4214148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 -- 172.21.15.90:0/2252624461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080094a0 con 0x7f42140b4940 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4213fff700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f42141489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f4218ae2700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 0x7f42140cb750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4208016000 tx=0x7f4208004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420801a7e0 con 0x7f42140b4940 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420801a940 con 0x7f42140b4940 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420801acf0 con 0x7f42140b4940 2023-11-28T11:08:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42140cf0f0 con 0x7f42140b4940 2023-11-28T11:08:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42140cf6e0 con 0x7f42140b4940 2023-11-28T11:08:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f420800f050 con 0x7f42140b4940 2023-11-28T11:08:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.904+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f4208024420 con 0x7f42140b4940 2023-11-28T11:08:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.905+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42140b8940 con 0x7f42140b4940 2023-11-28T11:08:19.913 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.909+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4208029070 con 0x7f42140b4940 2023-11-28T11:08:19.950 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:19.949+0000 7fa57ec07200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:08:19.994 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:19.992+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d04ea464-897d-4c86-b799-f04692ae0f16", "id": 5} v 0) v1 -- 0x7f4214155510 con 0x7f42140b4940 2023-11-28T11:08:20.005 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.003+0000 7f42117fa700 1 -- 172.21.15.90:0/2252624461 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d04ea464-897d-4c86-b799-f04692ae0f16", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4214155510 con 0x7f42140b4940 2023-11-28T11:08:20.005 INFO:teuthology.orchestra.run.smithi090.stdout:5 2023-11-28T11:08:20.006 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.004+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 msgr2=0x7f42140cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.004+0000 7f421ad46700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 0x7f42140cb750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4208016000 tx=0x7f4208004750 comp rx=0 tx=0).stop 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 shutdown_connections 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42140d2880 0x7f4214148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42140b4940 0x7f42140cb750 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 --2- 172.21.15.90:0/2252624461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f42140b8570 0x7f42141489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 >> 172.21.15.90:0/2252624461 conn(0x7f42140d30a0 msgr2=0x7f42140b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 shutdown_connections 2023-11-28T11:08:20.007 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.005+0000 7f421ad46700 1 -- 172.21.15.90:0/2252624461 wait complete. 2023-11-28T11:08:20.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 9a9bb320-9fc5-4038-a54a-93ca2fe9200b 6 2023-11-28T11:08:20.027 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:20.026+0000 7f9fe51c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:08:20.140 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.139+0000 7f36a8809700 1 Processor -- start 2023-11-28T11:08:20.141 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.139+0000 7f36a8809700 1 -- start start 2023-11-28T11:08:20.143 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a8809700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36a40d0560 0x7f36a40ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.143 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a8809700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f36a40cd4f0 0x7f36a40b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.143 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a8809700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a40b3660 con 0x7f36a4158040 2023-11-28T11:08:20.143 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a8809700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a40cea60 con 0x7f36a40d0560 2023-11-28T11:08:20.144 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a8809700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a40d0a00 con 0x7f36a40cd4f0 2023-11-28T11:08:20.144 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f36a40cd4f0 0x7f36a40b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.144 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f36a40cd4f0 0x7f36a40b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49064/0 (socket says 172.21.15.90:49064) 2023-11-28T11:08:20.144 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 -- 172.21.15.90:0/448499770 learned_addr learned my addr 172.21.15.90:0/448499770 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:20.144 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a259c700 1 --1- 172.21.15.90:0/448499770 >> v1:172.21.15.90:6789/0 conn(0x7f36a4158040 0x7f36a415b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:55296/0 (socket says 172.21.15.90:55296) 2023-11-28T11:08:20.145 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a1d9b700 1 --2- 172.21.15.90:0/448499770 >> v2:172.21.15.83:3300/0 conn(0x7f36a40d0560 0x7f36a40ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.145 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 -- 172.21.15.90:0/448499770 >> v2:172.21.15.83:3300/0 conn(0x7f36a40d0560 msgr2=0x7f36a40ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:20.145 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 --2- 172.21.15.90:0/448499770 >> v2:172.21.15.83:3300/0 conn(0x7f36a40d0560 0x7f36a40ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.145 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 -- 172.21.15.90:0/448499770 >> v1:172.21.15.90:6789/0 conn(0x7f36a4158040 legacy=0x7f36a415b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.145 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 -- 172.21.15.90:0/448499770 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40cb450 con 0x7f36a40cd4f0 2023-11-28T11:08:20.145 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a2d9d700 1 --2- 172.21.15.90:0/448499770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a40cd4f0 0x7f36a40b27e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f36a406ee00 tx=0x7f369c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b053e7b5e7d8f28 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.146 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a159a700 1 -- 172.21.15.90:0/448499770 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00e070 con 0x7f36a40cd4f0 2023-11-28T11:08:20.146 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a159a700 1 -- 172.21.15.90:0/448499770 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00b910 con 0x7f36a40cd4f0 2023-11-28T11:08:20.146 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.140+0000 7f36a159a700 1 -- 172.21.15.90:0/448499770 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00bcc0 con 0x7f36a40cd4f0 2023-11-28T11:08:20.146 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 -- 172.21.15.90:0/448499770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a40cd4f0 msgr2=0x7f36a40b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.146 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 --2- 172.21.15.90:0/448499770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a40cd4f0 0x7f36a40b27e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f36a406ee00 tx=0x7f369c00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:20.147 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 -- 172.21.15.90:0/448499770 shutdown_connections 2023-11-28T11:08:20.147 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 --2- 172.21.15.90:0/448499770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a40cd4f0 0x7f36a40b27e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.147 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 --2- 172.21.15.90:0/448499770 >> v2:172.21.15.83:3300/0 conn(0x7f36a40d0560 0x7f36a40ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.147 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 -- 172.21.15.90:0/448499770 >> 172.21.15.90:0/448499770 conn(0x7f36a406de20 msgr2=0x7f36a406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.147 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 -- 172.21.15.90:0/448499770 shutdown_connections 2023-11-28T11:08:20.148 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.141+0000 7f36a8809700 1 -- 172.21.15.90:0/448499770 wait complete. 2023-11-28T11:08:20.151 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.142+0000 7f36a8809700 1 Processor -- start 2023-11-28T11:08:20.151 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 -- start start 2023-11-28T11:08:20.151 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a40cd4f0 0x7f36a40ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.151 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 0x7f36a40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.151 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a4158040 0x7f36a40b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.152 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40cae10 con 0x7f36a40cd4f0 2023-11-28T11:08:20.152 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a41e4ef0 con 0x7f36a4158040 2023-11-28T11:08:20.152 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a8809700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40c65a0 con 0x7f36a40d0560 2023-11-28T11:08:20.152 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a1d9b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 0x7f36a40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.152 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a1d9b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 0x7f36a40c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44170/0 (socket says 172.21.15.90:44170) 2023-11-28T11:08:20.153 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a2d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a4158040 0x7f36a40b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.153 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a1d9b700 1 -- 172.21.15.90:0/2990111370 learned_addr learned my addr 172.21.15.90:0/2990111370 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:20.153 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.143+0000 7f36a259c700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a40cd4f0 0x7f36a40ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.153 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a1d9b700 1 -- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a4158040 msgr2=0x7f36a40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.153 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a1d9b700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a4158040 0x7f36a40b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.154 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a1d9b700 1 -- 172.21.15.90:0/2990111370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a40cd4f0 msgr2=0x7f36a40ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.154 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a1d9b700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a40cd4f0 0x7f36a40ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.154 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a1d9b700 1 -- 172.21.15.90:0/2990111370 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c009b30 con 0x7f36a40d0560 2023-11-28T11:08:20.154 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a2d9d700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a4158040 0x7f36a40b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:20.154 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a259c700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a40cd4f0 0x7f36a40ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:20.154 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a1d9b700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 0x7f36a40c7080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f369800e980 tx=0x7f369800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.155 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698009710 con 0x7f36a40d0560 2023-11-28T11:08:20.155 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369800c690 con 0x7f36a40d0560 2023-11-28T11:08:20.155 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800ca40 con 0x7f36a40d0560 2023-11-28T11:08:20.155 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a4158410 con 0x7f36a40d0560 2023-11-28T11:08:20.155 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a40cb730 con 0x7f36a40d0560 2023-11-28T11:08:20.156 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3698019030 con 0x7f36a40d0560 2023-11-28T11:08:20.156 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.144+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f3698006700 con 0x7f36a40d0560 2023-11-28T11:08:20.156 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.145+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a40cba10 con 0x7f36a40d0560 2023-11-28T11:08:20.156 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.147+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3698021060 con 0x7f36a40d0560 2023-11-28T11:08:20.234 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.230+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9a9bb320-9fc5-4038-a54a-93ca2fe9200b", "id": 6} v 0) v1 -- 0x7f36a415a9c0 con 0x7f36a40d0560 2023-11-28T11:08:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.238+0000 7f3692ffd700 1 -- 172.21.15.90:0/2990111370 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a9bb320-9fc5-4038-a54a-93ca2fe9200b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3698017070 con 0x7f36a40d0560 2023-11-28T11:08:20.239 INFO:teuthology.orchestra.run.smithi090.stdout:6 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 msgr2=0x7f36a40c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 0x7f36a40c7080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f369800e980 tx=0x7f369800ec90 comp rx=0 tx=0).stop 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 shutdown_connections 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36a4158040 0x7f36a40b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a40cd4f0 0x7f36a40ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 --2- 172.21.15.90:0/2990111370 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36a40d0560 0x7f36a40c7080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 >> 172.21.15.90:0/2990111370 conn(0x7f36a406de20 msgr2=0x7f36a415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 shutdown_connections 2023-11-28T11:08:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.239+0000 7f36a8809700 1 -- 172.21.15.90:0/2990111370 wait complete. 2023-11-28T11:08:20.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 646d177e-e2dc-4ed0-95b1-724a20d6aca4 7 2023-11-28T11:08:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.412+0000 7fe68b54e700 1 Processor -- start 2023-11-28T11:08:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.412+0000 7fe68b54e700 1 -- start start 2023-11-28T11:08:20.416 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe68b54e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 0x7fe6840aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe68b54e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 0x7fe68406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe68b54e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe68406e1a0 0x7fe6841604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe68b54e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68406e620 con 0x7fe68406e1a0 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe68b54e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840b5020 con 0x7fe68415c890 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe68b54e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68406d4a0 con 0x7fe6840b4bb0 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe6892ea700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 0x7fe6840aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe688ae9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 0x7fe68406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe6892ea700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 0x7fe6840aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49072/0 (socket says 172.21.15.90:49072) 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe688ae9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 0x7fe68406dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44176/0 (socket says 172.21.15.90:44176) 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe6892ea700 1 -- 172.21.15.90:0/284794797 learned_addr learned my addr 172.21.15.90:0/284794797 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.413+0000 7fe689aeb700 1 --2- 172.21.15.90:0/284794797 >> v2:172.21.15.83:3300/0 conn(0x7fe68406e1a0 0x7fe6841604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe688ae9700 1 -- 172.21.15.90:0/284794797 >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 msgr2=0x7fe6840aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe688ae9700 1 --2- 172.21.15.90:0/284794797 >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 0x7fe6840aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe688ae9700 1 -- 172.21.15.90:0/284794797 >> v2:172.21.15.83:3300/0 conn(0x7fe68406e1a0 msgr2=0x7fe6841604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe688ae9700 1 --2- 172.21.15.90:0/284794797 >> v2:172.21.15.83:3300/0 conn(0x7fe68406e1a0 0x7fe6841604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe688ae9700 1 -- 172.21.15.90:0/284794797 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840b5ae0 con 0x7fe6840b4bb0 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe6892ea700 1 --2- 172.21.15.90:0/284794797 >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 0x7fe6840aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe688ae9700 1 --2- 172.21.15.90:0/284794797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 0x7fe68406dc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe674009840 tx=0x7fe674009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4aa7aac16418d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe67bfff700 1 -- 172.21.15.90:0/284794797 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674011830 con 0x7fe6840b4bb0 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe67bfff700 1 -- 172.21.15.90:0/284794797 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe674011990 con 0x7fe6840b4bb0 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe67bfff700 1 -- 172.21.15.90:0/284794797 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674011d20 con 0x7fe6840b4bb0 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe68b54e700 1 -- 172.21.15.90:0/284794797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 msgr2=0x7fe68406dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.414+0000 7fe68b54e700 1 --2- 172.21.15.90:0/284794797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 0x7fe68406dc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe674009840 tx=0x7fe674009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 -- 172.21.15.90:0/284794797 shutdown_connections 2023-11-28T11:08:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 --2- 172.21.15.90:0/284794797 >> v2:172.21.15.83:3300/0 conn(0x7fe68406e1a0 0x7fe6841604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.419 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 --2- 172.21.15.90:0/284794797 >> v2:172.21.15.90:3300/0 conn(0x7fe68415c890 0x7fe6840aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.419 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 --2- 172.21.15.90:0/284794797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe6840b4bb0 0x7fe68406dc60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.419 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 -- 172.21.15.90:0/284794797 >> 172.21.15.90:0/284794797 conn(0x7fe6840b2b60 msgr2=0x7fe6840b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.419 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 -- 172.21.15.90:0/284794797 shutdown_connections 2023-11-28T11:08:20.419 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.415+0000 7fe68b54e700 1 -- 172.21.15.90:0/284794797 wait complete. 2023-11-28T11:08:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.416+0000 7fe68b54e700 1 Processor -- start 2023-11-28T11:08:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.416+0000 7fe68b54e700 1 -- start start 2023-11-28T11:08:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.416+0000 7fe68b54e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe68406e1a0 0x7fe684146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.416+0000 7fe68b54e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6840b4bb0 0x7fe6840c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe68b54e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 0x7fe6840c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe68b54e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840c9f20 con 0x7fe6840b4bb0 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe68b54e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840ca170 con 0x7fe68415c890 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe68b54e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840ca450 con 0x7fe68406e1a0 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe6892ea700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe68406e1a0 0x7fe684146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 0x7fe6840c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 0x7fe6840c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49082/0 (socket says 172.21.15.90:49082) 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe6892ea700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe68406e1a0 0x7fe684146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:44180/0 (socket says 172.21.15.90:44180) 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 -- 172.21.15.90:0/75152348 learned_addr learned my addr 172.21.15.90:0/75152348 (peer_addr_for_me v2:172.21.15.90:0/0) 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe688ae9700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6840b4bb0 0x7fe6840c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 -- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe68406e1a0 msgr2=0x7fe684146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe68406e1a0 0x7fe684146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 -- 172.21.15.90:0/75152348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6840b4bb0 msgr2=0x7fe6840c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.423 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6840b4bb0 0x7fe6840c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 -- 172.21.15.90:0/75152348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680009500 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe689aeb700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 0x7fe6840c92b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00eed0 tx=0x7fe67c00ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c010040 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00b9f0 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6740094a0 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00bda0 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.417+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6840b5240 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.418+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe67c01d400 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.418+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fe67c014b00 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.419+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68406c400 con 0x7fe68415c890 2023-11-28T11:08:20.424 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.421+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe67c018020 con 0x7fe68415c890 2023-11-28T11:08:20.509 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:20.507+0000 7fa57ec07200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:08:20.553 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.551+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "646d177e-e2dc-4ed0-95b1-724a20d6aca4", "id": 7} v 0) v1 -- 0x7fe6840b5520 con 0x7fe68415c890 2023-11-28T11:08:20.563 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.562+0000 7fe679ffb700 1 -- 172.21.15.90:0/75152348 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "646d177e-e2dc-4ed0-95b1-724a20d6aca4", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe67c0144a0 con 0x7fe68415c890 2023-11-28T11:08:20.563 INFO:teuthology.orchestra.run.smithi090.stdout:7 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 msgr2=0x7fe6840c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 0x7fe6840c92b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00eed0 tx=0x7fe67c00ef00 comp rx=0 tx=0).stop 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 shutdown_connections 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68415c890 0x7fe6840c92b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe68406e1a0 0x7fe684146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 --2- 172.21.15.90:0/75152348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6840b4bb0 0x7fe6840c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 >> 172.21.15.90:0/75152348 conn(0x7fe6840b2b60 msgr2=0x7fe6840aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 shutdown_connections 2023-11-28T11:08:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2023-11-28T11:08:20.563+0000 7fe68b54e700 1 -- 172.21.15.90:0/75152348 wait complete. 2023-11-28T11:08:20.576 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T11:08:20.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T11:08:20.577 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:20.576+0000 7fa57ec07200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:08:20.578 INFO:tasks.ceph.osd.0:Started 2023-11-28T11:08:20.579 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T11:08:20.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T11:08:20.580 INFO:tasks.ceph.osd.1:Started 2023-11-28T11:08:20.580 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T11:08:20.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T11:08:20.582 INFO:tasks.ceph.osd.2:Started 2023-11-28T11:08:20.583 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T11:08:20.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T11:08:20.584 INFO:tasks.ceph.osd.3:Started 2023-11-28T11:08:20.585 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T11:08:20.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T11:08:20.587 INFO:tasks.ceph.osd.4:Started 2023-11-28T11:08:20.587 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T11:08:20.587 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T11:08:20.589 INFO:tasks.ceph.osd.5:Started 2023-11-28T11:08:20.589 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T11:08:20.589 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T11:08:20.591 INFO:tasks.ceph.osd.6:Started 2023-11-28T11:08:20.591 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T11:08:20.591 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T11:08:20.595 INFO:tasks.ceph.osd.7:Started 2023-11-28T11:08:20.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:20.641 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:20.640+0000 7f9fe51c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:08:20.725 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:20.723+0000 7f9fe51c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:08:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.750+0000 7f3b167f0700 1 Processor -- start 2023-11-28T11:08:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.750+0000 7f3b167f0700 1 -- start start 2023-11-28T11:08:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b167f0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b6780 0x7f3b100cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b167f0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b100b2f20 0x7f3b100ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b167f0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1015f180 con 0x7f3b100afd90 2023-11-28T11:08:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b167f0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1015ec90 con 0x7f3b100b2f20 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b167f0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100cb510 con 0x7f3b100b6780 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b100b2f20 0x7f3b100ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b100b2f20 0x7f3b100ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47590/0 (socket says 172.21.15.83:47590) 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 -- 172.21.15.83:0/2402187793 learned_addr learned my addr 172.21.15.83:0/2402187793 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b0ffff700 1 --2- 172.21.15.83:0/2402187793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b6780 0x7f3b100cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 -- 172.21.15.83:0/2402187793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b6780 msgr2=0x7f3b100cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 --2- 172.21.15.83:0/2402187793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b6780 0x7f3b100cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b0effd700 1 -- 172.21.15.83:0/2402187793 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393566529 0 0) 0x7f3b1015f180 con 0x7f3b100afd90 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 -- 172.21.15.83:0/2402187793 >> v1:172.21.15.90:6789/0 conn(0x7f3b100afd90 legacy=0x7f3b100b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 -- 172.21.15.83:0/2402187793 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1015ab70 con 0x7f3b100b2f20 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.752+0000 7f3b14d8d700 1 --2- 172.21.15.83:0/2402187793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100b2f20 0x7f3b100ce960 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8a215f4f03dfcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b0effd700 1 -- 172.21.15.83:0/2402187793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800e070 con 0x7f3b100b2f20 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b0effd700 1 -- 172.21.15.83:0/2402187793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0800b910 con 0x7f3b100b2f20 2023-11-28T11:08:20.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b0effd700 1 -- 172.21.15.83:0/2402187793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800bcc0 con 0x7f3b100b2f20 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 -- 172.21.15.83:0/2402187793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100b2f20 msgr2=0x7f3b100ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 --2- 172.21.15.83:0/2402187793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100b2f20 0x7f3b100ce960 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 -- 172.21.15.83:0/2402187793 shutdown_connections 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 --2- 172.21.15.83:0/2402187793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100b2f20 0x7f3b100ce960 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 --2- 172.21.15.83:0/2402187793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b6780 0x7f3b100cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 -- 172.21.15.83:0/2402187793 >> 172.21.15.83:0/2402187793 conn(0x7f3b1006ff50 msgr2=0x7f3b100cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 -- 172.21.15.83:0/2402187793 shutdown_connections 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.753+0000 7f3b167f0700 1 -- 172.21.15.83:0/2402187793 wait complete. 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.754+0000 7f3b167f0700 1 Processor -- start 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.754+0000 7f3b167f0700 1 -- start start 2023-11-28T11:08:20.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b167f0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 0x7f3b1006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b167f0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b2f20 0x7f3b1014d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b167f0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b100b6780 0x7f3b100b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b167f0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10155bd0 con 0x7f3b100afd90 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b167f0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1014d600 con 0x7f3b100b6780 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b167f0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1006c1f0 con 0x7f3b100b2f20 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b0ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 0x7f3b1006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b0ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 0x7f3b1006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47592/0 (socket says 172.21.15.83:47592) 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b0ffff700 1 -- 172.21.15.83:0/3702833632 learned_addr learned my addr 172.21.15.83:0/3702833632 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b14d8d700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b100b6780 0x7f3b100b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.755+0000 7f3b0f7fe700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b2f20 0x7f3b1014d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0ffff700 1 -- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b2f20 msgr2=0x7f3b1014d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0ffff700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b2f20 0x7f3b1014d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0ffff700 1 -- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b100b6780 msgr2=0x7f3b100b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0ffff700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b100b6780 0x7f3b100b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0ffff700 1 -- 172.21.15.83:0/3702833632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009b30 con 0x7f3b100afd90 2023-11-28T11:08:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0ffff700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 0x7f3b1006bc10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000e9b0 tx=0x7f3b0000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00009710 con 0x7f3b100afd90 2023-11-28T11:08:20.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b100cb7b0 con 0x7f3b100afd90 2023-11-28T11:08:20.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0000ee50 con 0x7f3b100afd90 2023-11-28T11:08:20.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000ca50 con 0x7f3b100afd90 2023-11-28T11:08:20.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1015eea0 con 0x7f3b100afd90 2023-11-28T11:08:20.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3b00019030 con 0x7f3b100afd90 2023-11-28T11:08:20.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.756+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3b000068d0 con 0x7f3b100afd90 2023-11-28T11:08:20.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.757+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b100b04e0 con 0x7f3b100afd90 2023-11-28T11:08:20.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.759+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3b00021060 con 0x7f3b100afd90 2023-11-28T11:08:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.829+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b100cc190 con 0x7f3b100afd90 2023-11-28T11:08:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.829+0000 7f3b0cff9700 1 -- 172.21.15.83:0/3702833632 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3b00017070 con 0x7f3b100afd90 2023-11-28T11:08:20.831 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:20.831 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:20.553982+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 msgr2=0x7f3b1006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 0x7f3b1006bc10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000e9b0 tx=0x7f3b0000ecc0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 shutdown_connections 2023-11-28T11:08:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b100b6780 0x7f3b100b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b100afd90 0x7f3b1006bc10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 --2- 172.21.15.83:0/3702833632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b100b2f20 0x7f3b1014d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 >> 172.21.15.83:0/3702833632 conn(0x7f3b1006ff50 msgr2=0x7f3b1015e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 shutdown_connections 2023-11-28T11:08:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:20.830+0000 7f3b167f0700 1 -- 172.21.15.83:0/3702833632 wait complete. 2023-11-28T11:08:20.844 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T11:08:20.844 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T11:08:20.953 INFO:tasks.ceph.osd.6.smithi090.stderr:2023-11-28T11:08:20.951+0000 7f9bb0fba640 -1 Falling back to public interface 2023-11-28T11:08:21.197 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-28T11:08:21.195+0000 7f779f77a640 -1 Falling back to public interface 2023-11-28T11:08:21.197 INFO:tasks.ceph.osd.7.smithi090.stderr:2023-11-28T11:08:21.195+0000 7fe433159640 -1 Falling back to public interface 2023-11-28T11:08:21.210 INFO:tasks.ceph.osd.4.smithi090.stderr:2023-11-28T11:08:21.208+0000 7efe1cd6e640 -1 Falling back to public interface 2023-11-28T11:08:21.213 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-28T11:08:21.212+0000 7f5954d4a640 -1 Falling back to public interface 2023-11-28T11:08:21.214 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-28T11:08:21.212+0000 7f9bf1a91640 -1 Falling back to public interface 2023-11-28T11:08:21.221 INFO:tasks.ceph.osd.5.smithi090.stderr:2023-11-28T11:08:21.219+0000 7f0e7e947640 -1 Falling back to public interface 2023-11-28T11:08:21.223 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-28T11:08:21.222+0000 7fb4d27f3640 -1 Falling back to public interface 2023-11-28T11:08:21.298 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:21.297+0000 7fa57ec07200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:21.374 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:21.373+0000 7fa57ec07200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:08:21.402 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:21.400+0000 7f9fe51c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:21.479 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:21.477+0000 7f9fe51c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:08:21.514 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:21.512+0000 7fa57ec07200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:21.620 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:21.618+0000 7f9fe51c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:21.651 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:21.768 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.766+0000 7f84845c0700 1 Processor -- start 2023-11-28T11:08:21.768 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.766+0000 7f84845c0700 1 -- start start 2023-11-28T11:08:21.774 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f84845c0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.774 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f84845c0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f847c0cf9b0 0x7f847c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.774 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f84845c0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0d0370 con 0x7f847c0b53d0 2023-11-28T11:08:21.774 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f84845c0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c0d3ed0 con 0x7f847c0cf9b0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f84845c0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0b63a0 con 0x7f847c15c8f0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f8482b5d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f847c0cf9b0 0x7f847c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f8482b5d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f847c0cf9b0 0x7f847c0cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47602/0 (socket says 172.21.15.83:47602) 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f8482b5d700 1 -- 172.21.15.83:0/3450060224 learned_addr learned my addr 172.21.15.83:0/3450060224 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.767+0000 7f848235c700 1 --2- 172.21.15.83:0/3450060224 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f8482b5d700 1 -- 172.21.15.83:0/3450060224 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 msgr2=0x7f847c0b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f8482b5d700 1 --2- 172.21.15.83:0/3450060224 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f8482b5d700 1 -- 172.21.15.83:0/3450060224 >> v1:172.21.15.90:6789/0 conn(0x7f847c0b53d0 legacy=0x7f847c0d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f8482b5d700 1 -- 172.21.15.83:0/3450060224 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c0d0da0 con 0x7f847c0cf9b0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f8482b5d700 1 --2- 172.21.15.83:0/3450060224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c0cfe30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84780098f0 tx=0x7f8478009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a56a2baf28b4df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f848135a700 1 -- 172.21.15.83:0/3450060224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478011830 con 0x7f847c0cf9b0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f848135a700 1 -- 172.21.15.83:0/3450060224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478011990 con 0x7f847c0cf9b0 2023-11-28T11:08:21.775 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f848135a700 1 -- 172.21.15.83:0/3450060224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478011d20 con 0x7f847c0cf9b0 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f84845c0700 1 -- 172.21.15.83:0/3450060224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 msgr2=0x7f847c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.768+0000 7f84845c0700 1 --2- 172.21.15.83:0/3450060224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c0cfe30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84780098f0 tx=0x7f8478009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.769+0000 7f84845c0700 1 -- 172.21.15.83:0/3450060224 shutdown_connections 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.769+0000 7f84845c0700 1 --2- 172.21.15.83:0/3450060224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c0cfe30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.769+0000 7f84845c0700 1 --2- 172.21.15.83:0/3450060224 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.769+0000 7f84845c0700 1 -- 172.21.15.83:0/3450060224 >> 172.21.15.83:0/3450060224 conn(0x7f847c0719a0 msgr2=0x7f847c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.769+0000 7f84845c0700 1 -- 172.21.15.83:0/3450060224 shutdown_connections 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.769+0000 7f84845c0700 1 -- 172.21.15.83:0/3450060224 wait complete. 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.770+0000 7f84845c0700 1 Processor -- start 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.770+0000 7f84845c0700 1 -- start start 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f847c0b53d0 0x7f847c14e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c1f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.776 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0cb2f0 con 0x7f847c0cf9b0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0c9a40 con 0x7f847c0b53d0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0c91a0 con 0x7f847c15c8f0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c1f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c1f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47606/0 (socket says 172.21.15.83:47606) 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 -- 172.21.15.83:0/3943059081 learned_addr learned my addr 172.21.15.83:0/3943059081 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8482b5d700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f848235c700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f847c0b53d0 0x7f847c14e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 -- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 msgr2=0x7f847c0c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 -- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f847c0b53d0 msgr2=0x7f847c14e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f847c0b53d0 0x7f847c14e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 -- 172.21.15.83:0/3943059081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478009550 con 0x7f847c0cf9b0 2023-11-28T11:08:21.777 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8481b5b700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c1f2bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c4e0 tx=0x7f846c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0052d0 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c005430 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c0cda50 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0038c0 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.771+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c0cd770 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.772+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f846c019030 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.772+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f846c01b3b0 con 0x7f847c0cf9b0 2023-11-28T11:08:21.778 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.772+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c0d1040 con 0x7f847c0cf9b0 2023-11-28T11:08:21.779 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.774+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f846c017070 con 0x7f847c0cf9b0 2023-11-28T11:08:21.817 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:21.816+0000 7fa57ec07200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:08:21.846 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.845+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f847c074240 con 0x7f847c0cf9b0 2023-11-28T11:08:21.847 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.845+0000 7f8472ffd700 1 -- 172.21.15.83:0/3943059081 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f846c003a20 con 0x7f847c0cf9b0 2023-11-28T11:08:21.847 INFO:teuthology.misc.health.smithi083.stdout: 2023-11-28T11:08:21.847 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":10,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:20.553982+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:21.848 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 msgr2=0x7f847c1f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.848 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c1f2bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c4e0 tx=0x7f846c00c7f0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.848 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 shutdown_connections 2023-11-28T11:08:21.849 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f847c15c8f0 0x7f847c0c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.849 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f847c0b53d0 0x7f847c14e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.849 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 --2- 172.21.15.83:0/3943059081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f847c0cf9b0 0x7f847c1f2bd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.849 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 >> 172.21.15.83:0/3943059081 conn(0x7f847c0719a0 msgr2=0x7f847c0d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.849 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 shutdown_connections 2023-11-28T11:08:21.849 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:21.846+0000 7f84845c0700 1 -- 172.21.15.83:0/3943059081 wait complete. 2023-11-28T11:08:21.859 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:08:21.919 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:21.917+0000 7f9fe51c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:08:21.953 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:21.951+0000 7fa57ec07200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:08:22.053 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:22.051+0000 7f9fe51c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:08:22.169 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:22.167+0000 7fa57ec07200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:08:22.269 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:22.267+0000 7f9fe51c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:08:22.323 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:22.321+0000 7fa57ec07200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:08:22.426 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:22.425+0000 7f9fe51c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:08:22.435 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:22.434+0000 7fa57ec07200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:08:22.539 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:22.538+0000 7f9fe51c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:08:22.750 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:22.749+0000 7fa57ec07200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:08:22.850 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:22.848+0000 7fa57ec07200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:22.860 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:22.858+0000 7f9fe51c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:08:22.960 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:22.958+0000 7f9fe51c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:23.031 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:23.030+0000 7fa57ec07200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:08:23.142 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:23.141+0000 7f9fe51c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:08:23.546 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:23.544+0000 7fa57ec07200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:08:23.666 INFO:tasks.ceph.mgr.x.smithi090.stderr:2023-11-28T11:08:23.665+0000 7f9fe51c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:08:24.536 INFO:tasks.ceph.osd.6.smithi090.stderr:2023-11-28T11:08:24.536+0000 7f9bb0fba640 -1 osd.6 0 log_to_monitors true 2023-11-28T11:08:24.570 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:24.568+0000 7fa535e6e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T11:08:24.573 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-28T11:08:24.572+0000 7fa535e6e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T11:08:24.722 INFO:tasks.ceph.osd.7.smithi090.stderr:2023-11-28T11:08:24.721+0000 7fe433159640 -1 osd.7 0 log_to_monitors true 2023-11-28T11:08:24.748 INFO:tasks.ceph.osd.4.smithi090.stderr:2023-11-28T11:08:24.747+0000 7efe1cd6e640 -1 osd.4 0 log_to_monitors true 2023-11-28T11:08:24.760 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-28T11:08:24.759+0000 7fb4d27f3640 -1 osd.2 0 log_to_monitors true 2023-11-28T11:08:24.767 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-28T11:08:24.766+0000 7f9bf1a91640 -1 osd.3 0 log_to_monitors true 2023-11-28T11:08:24.773 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-28T11:08:24.772+0000 7f5954d4a640 -1 osd.1 0 log_to_monitors true 2023-11-28T11:08:24.775 INFO:tasks.ceph.osd.5.smithi090.stderr:2023-11-28T11:08:24.775+0000 7f0e7e947640 -1 osd.5 0 log_to_monitors true 2023-11-28T11:08:24.776 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-28T11:08:24.775+0000 7f779f77a640 -1 osd.0 0 log_to_monitors true 2023-11-28T11:08:27.798 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-28T11:08:27.797+0000 7fb4c6bab700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T11:08:27.808 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-28T11:08:27.805+0000 7f7793b32700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T11:08:27.809 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-28T11:08:27.808+0000 7fb4be785700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:27.812 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-28T11:08:27.811+0000 7f5949102700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T11:08:27.814 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-28T11:08:27.813+0000 7f9be5e49700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T11:08:27.825 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-28T11:08:27.824+0000 7f778b70c700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:27.828 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-28T11:08:27.827+0000 7f5940cdc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:27.828 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-28T11:08:27.827+0000 7f9bdda23700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:27.848 INFO:tasks.ceph.osd.4.smithi090.stderr:2023-11-28T11:08:27.848+0000 7efe11126700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T11:08:27.869 INFO:tasks.ceph.osd.6.smithi090.stderr:2023-11-28T11:08:27.867+0000 7f9ba5372700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T11:08:27.876 INFO:tasks.ceph.osd.4.smithi090.stderr:2023-11-28T11:08:27.876+0000 7efe08d00700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:27.883 INFO:tasks.ceph.osd.7.smithi090.stderr:2023-11-28T11:08:27.882+0000 7fe427511700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T11:08:27.889 INFO:tasks.ceph.osd.6.smithi090.stderr:2023-11-28T11:08:27.889+0000 7f9b9cf4c700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:27.915 INFO:tasks.ceph.osd.7.smithi090.stderr:2023-11-28T11:08:27.915+0000 7fe41f0eb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:28.055 INFO:tasks.ceph.osd.5.smithi090.stderr:2023-11-28T11:08:28.055+0000 7f0e72cff700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T11:08:28.066 INFO:tasks.ceph.osd.5.smithi090.stderr:2023-11-28T11:08:28.066+0000 7f0e6a8d9700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:28.666 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:28.788 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.786+0000 7fd20fd8a700 1 Processor -- start 2023-11-28T11:08:28.788 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.787+0000 7fd20fd8a700 1 -- start start 2023-11-28T11:08:28.789 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.787+0000 7fd20fd8a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd20806d3d0 0x7fd20806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.789 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.787+0000 7fd20fd8a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20815d8c0 con 0x7fd2080b0690 2023-11-28T11:08:28.789 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.787+0000 7fd20fd8a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20815d030 con 0x7fd208158d90 2023-11-28T11:08:28.789 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.787+0000 7fd20fd8a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2080cdb00 con 0x7fd20806d3d0 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20d325700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd20806d3d0 0x7fd20806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20e327700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd2080b0690 0x7fd2080cdf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45572/0 (socket says 172.21.15.83:45572) 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20e327700 1 -- 172.21.15.83:0/4042215066 learned_addr learned my addr 172.21.15.83:0/4042215066 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20d325700 1 -- 172.21.15.83:0/4042215066 >> v1:172.21.15.90:6789/0 conn(0x7fd208158d90 legacy=0x7fd20815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20d325700 1 -- 172.21.15.83:0/4042215066 >> v1:172.21.15.83:6789/0 conn(0x7fd2080b0690 legacy=0x7fd2080cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20d325700 1 -- 172.21.15.83:0/4042215066 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20815d300 con 0x7fd20806d3d0 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20cb24700 1 -- 172.21.15.83:0/4042215066 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299468428 0 0) 0x7fd20815d8c0 con 0x7fd2080b0690 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20d325700 1 --2- 172.21.15.83:0/4042215066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd20806d3d0 0x7fd20806d870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd20400bb10 tx=0x7fd20400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d3c476e0c05b8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20cb24700 1 -- 172.21.15.83:0/4042215066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400dcf0 con 0x7fd20806d3d0 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20cb24700 1 -- 172.21.15.83:0/4042215066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20400de50 con 0x7fd20806d3d0 2023-11-28T11:08:28.790 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.788+0000 7fd20cb24700 1 -- 172.21.15.83:0/4042215066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204010690 con 0x7fd20806d3d0 2023-11-28T11:08:28.791 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 -- 172.21.15.83:0/4042215066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd20806d3d0 msgr2=0x7fd20806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/4042215066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd20806d3d0 0x7fd20806d870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd20400bb10 tx=0x7fd20400be20 comp rx=0 tx=0).stop 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 -- 172.21.15.83:0/4042215066 shutdown_connections 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/4042215066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd20806d3d0 0x7fd20806d870 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 -- 172.21.15.83:0/4042215066 >> 172.21.15.83:0/4042215066 conn(0x7fd20806af00 msgr2=0x7fd2080cb690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 -- 172.21.15.83:0/4042215066 shutdown_connections 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.789+0000 7fd20fd8a700 1 -- 172.21.15.83:0/4042215066 wait complete. 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.790+0000 7fd20fd8a700 1 Processor -- start 2023-11-28T11:08:28.792 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.790+0000 7fd20fd8a700 1 -- start start 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd20806d3d0 0x7fd2080b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2080b0690 0x7fd208159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 0x7fd208152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081ef0d0 con 0x7fd208158d90 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081ee1d0 con 0x7fd20806d3d0 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081ee450 con 0x7fd2080b0690 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 0x7fd208152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 0x7fd208152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47718/0 (socket says 172.21.15.83:47718) 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 -- 172.21.15.83:0/556807376 learned_addr learned my addr 172.21.15.83:0/556807376 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20d325700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2080b0690 0x7fd208159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.795 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20db26700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd20806d3d0 0x7fd2080b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 -- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2080b0690 msgr2=0x7fd208159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2080b0690 0x7fd208159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 -- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd20806d3d0 msgr2=0x7fd2080b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd20806d3d0 0x7fd2080b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 -- 172.21.15.83:0/556807376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20400b740 con 0x7fd208158d90 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20d325700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2080b0690 0x7fd208159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20e327700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 0x7fd208152100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd20000cba0 tx=0x7fd20000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20000ab80 con 0x7fd208158d90 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20000ad00 con 0x7fd208158d90 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.791+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2080cd720 con 0x7fd208158d90 2023-11-28T11:08:28.796 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.792+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200007310 con 0x7fd208158d90 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.792+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20815d5e0 con 0x7fd208158d90 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.792+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208159160 con 0x7fd208158d90 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.793+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fd200010070 con 0x7fd208158d90 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.793+0000 7fd1fe7fc700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1ec0682a0 0x7fd1ec06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.793+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd200084700 con 0x7fd208158d90 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.794+0000 7fd20db26700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1ec0682a0 0x7fd1ec06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.797 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.794+0000 7fd20db26700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1ec0682a0 0x7fd1ec06a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007fd0 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.799 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.798+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd200051e40 con 0x7fd208158d90 2023-11-28T11:08:28.993 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.992+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd208074240 con 0x7fd208158d90 2023-11-28T11:08:28.994 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.993+0000 7fd1fe7fc700 1 -- 172.21.15.83:0/556807376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10488 (secure 0 0 0) 0x7fd200018020 con 0x7fd208158d90 2023-11-28T11:08:28.994 INFO:teuthology.misc.health.smithi083.stdout: 2023-11-28T11:08:28.994 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":13,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:28.805065+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:28.996 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1ec0682a0 msgr2=0x7fd1ec06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.996 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1ec0682a0 0x7fd1ec06a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007fd0 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 msgr2=0x7fd208152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 0x7fd208152100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd20000cba0 tx=0x7fd20000cf60 comp rx=0 tx=0).stop 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 shutdown_connections 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1ec0682a0 0x7fd1ec06a750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd208158d90 0x7fd208152100 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd20806d3d0 0x7fd2080b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 --2- 172.21.15.83:0/556807376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2080b0690 0x7fd208159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 >> 172.21.15.83:0/556807376 conn(0x7fd20806af00 msgr2=0x7fd2081535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 shutdown_connections 2023-11-28T11:08:28.997 INFO:teuthology.misc.health.smithi083.stderr:2023-11-28T11:08:28.995+0000 7fd20fd8a700 1 -- 172.21.15.83:0/556807376 wait complete. 2023-11-28T11:08:29.007 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T11:08:29.007 INFO:tasks.ceph:Creating RBD pool 2023-11-28T11:08:29.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T11:08:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 Processor -- start 2023-11-28T11:08:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 -- start start 2023-11-28T11:08:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c8156d70 0x7f78c80d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78c80d0bf0 0x7f78c815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c8157140 con 0x7f78c815d1a0 2023-11-28T11:08:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c815dcc0 con 0x7f78c80d0bf0 2023-11-28T11:08:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78ce3f6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c815d9e0 con 0x7f78c8156d70 2023-11-28T11:08:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.126+0000 7f78c77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78c80d0bf0 0x7f78c815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78c80d0bf0 0x7f78c815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47840/0 (socket says 172.21.15.83:47840) 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 -- 172.21.15.83:0/3616381985 learned_addr learned my addr 172.21.15.83:0/3616381985 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 -- 172.21.15.83:0/3616381985 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c8156d70 msgr2=0x7f78c80d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c7fff700 1 --2- 172.21.15.83:0/3616381985 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c8156d70 0x7f78c80d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 --2- 172.21.15.83:0/3616381985 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c8156d70 0x7f78c80d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 -- 172.21.15.83:0/3616381985 >> v1:172.21.15.90:6789/0 conn(0x7f78c815d1a0 legacy=0x7f78c8160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 -- 172.21.15.83:0/3616381985 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c806bdb0 con 0x7f78c80d0bf0 2023-11-28T11:08:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c77fe700 1 --2- 172.21.15.83:0/3616381985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c80d0bf0 0x7f78c815cc60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78b80098f0 tx=0x7f78b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27f0d5a7c91df88 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c6ffd700 1 -- 172.21.15.83:0/3616381985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b800b8a0 con 0x7f78c80d0bf0 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c6ffd700 1 -- 172.21.15.83:0/3616381985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b800ba00 con 0x7f78c80d0bf0 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.127+0000 7f78c6ffd700 1 -- 172.21.15.83:0/3616381985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b800bd90 con 0x7f78c80d0bf0 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 -- 172.21.15.83:0/3616381985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c80d0bf0 msgr2=0x7f78c815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/3616381985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c80d0bf0 0x7f78c815cc60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78b80098f0 tx=0x7f78b8009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 -- 172.21.15.83:0/3616381985 shutdown_connections 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/3616381985 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c8156d70 0x7f78c80d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/3616381985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c80d0bf0 0x7f78c815cc60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 -- 172.21.15.83:0/3616381985 >> 172.21.15.83:0/3616381985 conn(0x7f78c806ea50 msgr2=0x7f78c8070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 -- 172.21.15.83:0/3616381985 shutdown_connections 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.128+0000 7f78ce3f6700 1 -- 172.21.15.83:0/3616381985 wait complete. 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.129+0000 7f78ce3f6700 1 Processor -- start 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.129+0000 7f78ce3f6700 1 -- start start 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78c8156d70 0x7f78c80b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 0x7f78c81f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c80b4f00 0x7f78c815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81f2410 con 0x7f78c815d1a0 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81f26f0 con 0x7f78c8156d70 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81f29d0 con 0x7f78c80b4f00 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 0x7f78c81f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 0x7f78c81f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47848/0 (socket says 172.21.15.83:47848) 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 -- 172.21.15.83:0/1909059188 learned_addr learned my addr 172.21.15.83:0/1909059188 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c7fff700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78c8156d70 0x7f78c80b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78cc993700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c80b4f00 0x7f78c815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 -- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c80b4f00 msgr2=0x7f78c815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c80b4f00 0x7f78c815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 -- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78c8156d70 msgr2=0x7f78c80b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78c8156d70 0x7f78c80b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 -- 172.21.15.83:0/1909059188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8009550 con 0x7f78c815d1a0 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c77fe700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 0x7f78c81f6050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009130 tx=0x7f78b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b80036a0 con 0x7f78c815d1a0 2023-11-28T11:08:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b801e070 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c806c370 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.130+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003a40 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.131+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c806c090 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.131+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c80cf1a0 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.132+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f78b800f050 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.132+0000 7f78c4ff9700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f78b40682a0 0x7f78b406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.132+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f78b8020070 con 0x7f78c815d1a0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.132+0000 7f78c7fff700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f78b40682a0 0x7f78b406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.132+0000 7f78c7fff700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f78b40682a0 0x7f78b406a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78c8158450 tx=0x7f78bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.136+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f78b80539a0 con 0x7f78c815d1a0 2023-11-28T11:08:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.320+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f78c80742b0 con 0x7f78c815d1a0 2023-11-28T11:08:29.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.835+0000 7f78c4ff9700 1 -- 172.21.15.83:0/1909059188 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f78b80150a0 con 0x7f78c815d1a0 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f78b40682a0 msgr2=0x7f78b406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f78b40682a0 0x7f78b406a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78c8158450 tx=0x7f78bc008040 comp rx=0 tx=0).stop 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 msgr2=0x7f78c81f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 0x7f78c81f6050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009130 tx=0x7f78b8004750 comp rx=0 tx=0).stop 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 shutdown_connections 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78c80b4f00 0x7f78c815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78c8156d70 0x7f78c80b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f78b40682a0 0x7f78b406a750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 --2- 172.21.15.83:0/1909059188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78c815d1a0 0x7f78c81f6050 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 >> 172.21.15.83:0/1909059188 conn(0x7f78c80d0bf0 msgr2=0x7f78c806c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 shutdown_connections 2023-11-28T11:08:29.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.838+0000 7f78ce3f6700 1 -- 172.21.15.83:0/1909059188 wait complete. 2023-11-28T11:08:29.842 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'rbd' created 2023-11-28T11:08:29.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.972+0000 7f9d4223c700 1 Processor -- start 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.972+0000 7f9d4223c700 1 -- start start 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d4223c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9d3c0b6770 0x7f9d3c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d4223c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d3c15b510 0x7f9d3c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d4223c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c0cee70 0x7f9d3c15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d4223c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3c15e190 con 0x7f9d3c15b510 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d4223c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3c15b9a0 con 0x7f9d3c0b6770 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d4223c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0cb4e0 con 0x7f9d3c0cee70 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d3c15b510 0x7f9d3c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d3c15b510 0x7f9d3c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47902/0 (socket says 172.21.15.83:47902) 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 -- 172.21.15.83:0/330770407 learned_addr learned my addr 172.21.15.83:0/330770407 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3b7fe700 1 --2- 172.21.15.83:0/330770407 >> v2:172.21.15.90:3300/0 conn(0x7f9d3c0b6770 0x7f9d3c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3bfff700 1 --2- 172.21.15.83:0/330770407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c0cee70 0x7f9d3c15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 -- 172.21.15.83:0/330770407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c0cee70 msgr2=0x7f9d3c15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 --2- 172.21.15.83:0/330770407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c0cee70 0x7f9d3c15e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 -- 172.21.15.83:0/330770407 >> v2:172.21.15.90:3300/0 conn(0x7f9d3c0b6770 msgr2=0x7f9d3c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 --2- 172.21.15.83:0/330770407 >> v2:172.21.15.90:3300/0 conn(0x7f9d3c0b6770 0x7f9d3c0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.973+0000 7f9d3affd700 1 -- 172.21.15.83:0/330770407 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c0cb7c0 con 0x7f9d3c15b510 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d3affd700 1 --2- 172.21.15.83:0/330770407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c15b510 0x7f9d3c0ce930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c06fb40 tx=0x7f9d2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d059dc7f35d21fea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d3a7fc700 1 -- 172.21.15.83:0/330770407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00b870 con 0x7f9d3c15b510 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d3a7fc700 1 -- 172.21.15.83:0/330770407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c00b9d0 con 0x7f9d3c15b510 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d3a7fc700 1 -- 172.21.15.83:0/330770407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00bd60 con 0x7f9d3c15b510 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 -- 172.21.15.83:0/330770407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c15b510 msgr2=0x7f9d3c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 --2- 172.21.15.83:0/330770407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c15b510 0x7f9d3c0ce930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c06fb40 tx=0x7f9d2c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 -- 172.21.15.83:0/330770407 shutdown_connections 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 --2- 172.21.15.83:0/330770407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c0cee70 0x7f9d3c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 --2- 172.21.15.83:0/330770407 >> v2:172.21.15.90:3300/0 conn(0x7f9d3c0b6770 0x7f9d3c0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 --2- 172.21.15.83:0/330770407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c15b510 0x7f9d3c0ce930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 -- 172.21.15.83:0/330770407 >> 172.21.15.83:0/330770407 conn(0x7f9d3c06de20 msgr2=0x7f9d3c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 -- 172.21.15.83:0/330770407 shutdown_connections 2023-11-28T11:08:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.974+0000 7f9d4223c700 1 -- 172.21.15.83:0/330770407 wait complete. 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 Processor -- start 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 -- start start 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d3c0b6770 0x7f9d3c06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 0x7f9d3c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c15b510 0x7f9d3c14e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0b0d00 con 0x7f9d3c0cee70 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c06c310 con 0x7f9d3c0b6770 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d4223c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c06c5f0 con 0x7f9d3c15b510 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.976+0000 7f9d3affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 0x7f9d3c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 0x7f9d3c0b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47906/0 (socket says 172.21.15.83:47906) 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 -- 172.21.15.83:0/2524991235 learned_addr learned my addr 172.21.15.83:0/2524991235 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3b7fe700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d3c0b6770 0x7f9d3c06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3bfff700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c15b510 0x7f9d3c14e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 -- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c15b510 msgr2=0x7f9d3c14e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c15b510 0x7f9d3c14e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 -- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d3c0b6770 msgr2=0x7f9d3c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d3c0b6770 0x7f9d3c06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 -- 172.21.15.83:0/2524991235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009550 con 0x7f9d3c0cee70 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3b7fe700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d3c0b6770 0x7f9d3c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d3affd700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 0x7f9d3c0b07c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c00 tx=0x7f9d2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004540 con 0x7f9d3c0cee70 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c0036a0 con 0x7f9d3c0cee70 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c0d0910 con 0x7f9d3c0cee70 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c003a50 con 0x7f9d3c0cee70 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.977+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c0cbaa0 con 0x7f9d3c0cee70 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.978+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c0d0aa0 con 0x7f9d3c0cee70 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.978+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f9d2c00f050 con 0x7f9d3c0cee70 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.979+0000 7f9d33fff700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9d1c068270 0x7f9d1c06a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.979+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f9d2c087150 con 0x7f9d3c0cee70 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.979+0000 7f9d3b7fe700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9d1c068270 0x7f9d1c06a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.979+0000 7f9d3b7fe700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9d1c068270 0x7f9d1c06a720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0ca380 tx=0x7f9d24008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:29.983+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d2c053d60 con 0x7f9d3c0cee70 2023-11-28T11:08:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.170+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9d3c0742b0 con 0x7f9d3c0cee70 2023-11-28T11:08:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.837+0000 7f9d33fff700 1 -- 172.21.15.83:0/2524991235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9d2c057380 con 0x7f9d3c0cee70 2023-11-28T11:08:30.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9d1c068270 msgr2=0x7f9d1c06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9d1c068270 0x7f9d1c06a720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0ca380 tx=0x7f9d24008040 comp rx=0 tx=0).stop 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 msgr2=0x7f9d3c0b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 0x7f9d3c0b07c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c00 tx=0x7f9d2c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 shutdown_connections 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d3c15b510 0x7f9d3c14e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d3c0b6770 0x7f9d3c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9d1c068270 0x7f9d1c06a720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 --2- 172.21.15.83:0/2524991235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c0cee70 0x7f9d3c0b07c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 >> 172.21.15.83:0/2524991235 conn(0x7f9d3c06de20 msgr2=0x7f9d3c15c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 shutdown_connections 2023-11-28T11:08:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:30.841+0000 7f9d4223c700 1 -- 172.21.15.83:0/2524991235 wait complete. 2023-11-28T11:08:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T11:08:30.858 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T11:08:30.858 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:08:30.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:08:30.860 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:08:30.861 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:08:30.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:08:30.863 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:08:30.863 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:08:30.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:08:30.865 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:08:30.865 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:08:30.866 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:08:30.868 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:08:30.868 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T11:08:30.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:30.953 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-28T11:08:30.953 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-28T11:08:30.954 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-28T11:08:30.952+0000 7fce3cc2d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T11:08:30.954 INFO:tasks.ceph.mds.a.smithi083.stderr:2023-11-28T11:08:30.953+0000 7fbd831d3700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2023-11-28T11:08:30.956 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2023-11-28T11:08:30.957 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2023-11-28T11:08:31.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.013+0000 7fa171aad700 1 Processor -- start 2023-11-28T11:08:31.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.013+0000 7fa171aad700 1 -- start start 2023-11-28T11:08:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa171aad700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c1654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa171aad700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c0d0ba0 con 0x7fa16c0cfe40 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa171aad700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c0cdad0 con 0x7fa16c0b63f0 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa171aad700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cba60 con 0x7fa16c0cd670 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa16affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa16c0cfe40 0x7fa16c0ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45834/0 (socket says 172.21.15.83:45834) 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa16affd700 1 -- 172.21.15.83:0/1411123665 learned_addr learned my addr 172.21.15.83:0/1411123665 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.014+0000 7fa16bfff700 1 --2- 172.21.15.83:0/1411123665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c1654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154413259 0 0) 0x7fa160001110 con 0x7fa16c0cfe40 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1540035b0 con 0x7fa16c0cfe40 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925829297 0 0) 0x7fa15c001110 con 0x7fa16c0b63f0 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa160001110 con 0x7fa16c0b63f0 2023-11-28T11:08:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16bfff700 1 -- 172.21.15.83:0/1411123665 >> v1:172.21.15.90:6789/0 conn(0x7fa16c0b63f0 legacy=0x7fa16c0d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16bfff700 1 -- 172.21.15.83:0/1411123665 >> v1:172.21.15.83:6789/0 conn(0x7fa16c0cfe40 legacy=0x7fa16c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16bfff700 1 -- 172.21.15.83:0/1411123665 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c071700 con 0x7fa16c0cd670 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16bfff700 1 --2- 172.21.15.83:0/1411123665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c1654f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa164009dc0 tx=0x7fa16400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c9fab1afac9e18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400e070 con 0x7fa16c0cd670 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16400bb20 con 0x7fa16c0cd670 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.015+0000 7fa16a7fc700 1 -- 172.21.15.83:0/1411123665 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164012430 con 0x7fa16c0cd670 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 -- 172.21.15.83:0/1411123665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 msgr2=0x7fa16c1654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 --2- 172.21.15.83:0/1411123665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c1654f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa164009dc0 tx=0x7fa16400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 -- 172.21.15.83:0/1411123665 shutdown_connections 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 --2- 172.21.15.83:0/1411123665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c1654f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 -- 172.21.15.83:0/1411123665 >> 172.21.15.83:0/1411123665 conn(0x7fa16c06f300 msgr2=0x7fa16c06f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 -- 172.21.15.83:0/1411123665 shutdown_connections 2023-11-28T11:08:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.016+0000 7fa171aad700 1 -- 172.21.15.83:0/1411123665 wait complete. 2023-11-28T11:08:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.017+0000 7fa171aad700 1 Processor -- start 2023-11-28T11:08:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.017+0000 7fa171aad700 1 -- start start 2023-11-28T11:08:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa171aad700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa16c0b63f0 0x7fa16c1e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa171aad700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa171aad700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 0x7fa16c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa171aad700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0c7e80 con 0x7fa16c0cfe40 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa171aad700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0c80d0 con 0x7fa16c0b63f0 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa171aad700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0c83b0 con 0x7fa16c0cd670 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 0x7fa16c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 0x7fa16c0ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47982/0 (socket says 172.21.15.83:47982) 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 -- 172.21.15.83:0/33778207 learned_addr learned my addr 172.21.15.83:0/33778207 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16b7fe700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa16c0b63f0 0x7fa16c1e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16affd700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 -- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 msgr2=0x7fa16c0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c0ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 -- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa16c0b63f0 msgr2=0x7fa16c1e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa16c0b63f0 0x7fa16c1e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16bfff700 1 -- 172.21.15.83:0/33778207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164009a70 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.018+0000 7fa16affd700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.019+0000 7fa16bfff700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 0x7fa16c0ca540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa16400b370 tx=0x7fa164004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.019+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164012430 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.019+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164020070 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.019+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16401ba90 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.019+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c0cb780 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.019+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c0cb4a0 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.020+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c0cf4c0 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.020+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fa16401bbf0 con 0x7fa16c0cfe40 2023-11-28T11:08:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.021+0000 7fa153fff700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1580682a0 0x7fa15806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.021+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa164022070 con 0x7fa16c0cfe40 2023-11-28T11:08:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.021+0000 7fa16b7fe700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1580682a0 0x7fa15806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.021+0000 7fa16b7fe700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1580682a0 0x7fa15806a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa15c001370 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.027+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa164054ab0 con 0x7fa16c0cfe40 2023-11-28T11:08:31.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.231+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa16c074320 con 0x7fa16c0cfe40 2023-11-28T11:08:31.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.232+0000 7fa153fff700 1 -- 172.21.15.83:0/33778207 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14185 (secure 0 0 0) 0x7fa16402c090 con 0x7fa16c0cfe40 2023-11-28T11:08:31.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:31.234 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":15,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:30.831830+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1580682a0 msgr2=0x7fa15806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1580682a0 0x7fa15806a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa15c001370 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 msgr2=0x7fa16c0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 0x7fa16c0ca540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa16400b370 tx=0x7fa164004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 shutdown_connections 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1580682a0 0x7fa15806a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa16c0cfe40 0x7fa16c0ca540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.234+0000 7fa171aad700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa16c0b63f0 0x7fa16c1e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.235+0000 7fa171aad700 1 --2- 172.21.15.83:0/33778207 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa16c0cd670 0x7fa16c0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.235+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 >> 172.21.15.83:0/33778207 conn(0x7fa16c06f300 msgr2=0x7fa16c0b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.235+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 shutdown_connections 2023-11-28T11:08:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.235+0000 7fa171aad700 1 -- 172.21.15.83:0/33778207 wait complete. 2023-11-28T11:08:31.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:31.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.372+0000 7fd6cd889700 1 Processor -- start 2023-11-28T11:08:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.372+0000 7fd6cd889700 1 -- start start 2023-11-28T11:08:31.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6cd889700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 0x7fd6c80b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6cd889700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 0x7fd6c80cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6cd889700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80b5b00 con 0x7fd6c80aee20 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6cd889700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c806c800 con 0x7fd6c806c370 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6cd889700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c806cae0 con 0x7fd6c80af1f0 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6c67fc700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 0x7fd6c80b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6c77fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 0x7fd6c80cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.373+0000 7fd6c6ffd700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fd6c80aee20 0x7fd6c806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44200/0 (socket says 172.21.15.83:44200) 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c77fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 0x7fd6c80cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:60432/0 (socket says 172.21.15.83:60432) 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c6ffd700 1 -- 172.21.15.83:0/2770901504 learned_addr learned my addr 172.21.15.83:0/2770901504 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c67fc700 1 --2- 172.21.15.83:0/2770901504 >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 0x7fd6c80b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:34990/0 (socket says 172.21.15.83:34990) 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c77fe700 1 -- 172.21.15.83:0/2770901504 >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 msgr2=0x7fd6c80b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c77fe700 1 --2- 172.21.15.83:0/2770901504 >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 0x7fd6c80b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c77fe700 1 -- 172.21.15.83:0/2770901504 >> v1:172.21.15.90:6789/0 conn(0x7fd6c80aee20 legacy=0x7fd6c806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c77fe700 1 -- 172.21.15.83:0/2770901504 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c806cdc0 con 0x7fd6c80af1f0 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c67fc700 1 --2- 172.21.15.83:0/2770901504 >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 0x7fd6c80b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c77fe700 1 --2- 172.21.15.83:0/2770901504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 0x7fd6c80cc800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009d90 tx=0x7fd6c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cfba1638454ed6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c5ffb700 1 -- 172.21.15.83:0/2770901504 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c000e070 con 0x7fd6c80af1f0 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c5ffb700 1 -- 172.21.15.83:0/2770901504 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c000bb20 con 0x7fd6c80af1f0 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.374+0000 7fd6c5ffb700 1 -- 172.21.15.83:0/2770901504 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0012430 con 0x7fd6c80af1f0 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 -- 172.21.15.83:0/2770901504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 msgr2=0x7fd6c80cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 --2- 172.21.15.83:0/2770901504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 0x7fd6c80cc800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009d90 tx=0x7fd6c000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 -- 172.21.15.83:0/2770901504 shutdown_connections 2023-11-28T11:08:31.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 --2- 172.21.15.83:0/2770901504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80af1f0 0x7fd6c80cc800 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 --2- 172.21.15.83:0/2770901504 >> v2:172.21.15.90:3300/0 conn(0x7fd6c806c370 0x7fd6c80b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 -- 172.21.15.83:0/2770901504 >> 172.21.15.83:0/2770901504 conn(0x7fd6c80b2b60 msgr2=0x7fd6c80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 -- 172.21.15.83:0/2770901504 shutdown_connections 2023-11-28T11:08:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.375+0000 7fd6cd889700 1 -- 172.21.15.83:0/2770901504 wait complete. 2023-11-28T11:08:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.376+0000 7fd6cd889700 1 Processor -- start 2023-11-28T11:08:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.376+0000 7fd6cd889700 1 -- start start 2023-11-28T11:08:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 0x7fd6c814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80aee20 0x7fd6c8154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6c80af1f0 0x7fd6c81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c8154e00 con 0x7fd6c806c370 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815a920 con 0x7fd6c80af1f0 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80c69d0 con 0x7fd6c80aee20 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 0x7fd6c814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 0x7fd6c814e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48000/0 (socket says 172.21.15.83:48000) 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 -- 172.21.15.83:0/3083348866 learned_addr learned my addr 172.21.15.83:0/3083348866 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c77fe700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6c80af1f0 0x7fd6c81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c67fc700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80aee20 0x7fd6c8154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 -- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80aee20 msgr2=0x7fd6c8154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80aee20 0x7fd6c8154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 -- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6c80af1f0 msgr2=0x7fd6c81548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6c80af1f0 0x7fd6c81548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 -- 172.21.15.83:0/3083348866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b80094a0 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6c6ffd700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 0x7fd6c814e020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c550 tx=0x7fd6bc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc005dc0 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0070d0 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0009a70 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.377+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc007460 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.378+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c806fc70 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.378+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c806d0a0 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.379+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fd6bc019030 con 0x7fd6c806c370 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.379+0000 7fd6affff700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd6b40682a0 0x7fd6b406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.379+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd6bc0849e0 con 0x7fd6c806c370 2023-11-28T11:08:31.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.384+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6bc0515e0 con 0x7fd6c806c370 2023-11-28T11:08:31.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.386+0000 7fd6c67fc700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd6b40682a0 0x7fd6b406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.386+0000 7fd6c67fc700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd6b40682a0 0x7fd6b406a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80ae4d0 tx=0x7fd6b8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.574+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6c806d380 con 0x7fd6c806c370 2023-11-28T11:08:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.574+0000 7fd6affff700 1 -- 172.21.15.83:0/3083348866 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6bc0170a0 con 0x7fd6c806c370 2023-11-28T11:08:31.575 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:08:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd6b40682a0 msgr2=0x7fd6b406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd6b40682a0 0x7fd6b406a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80ae4d0 tx=0x7fd6b8009210 comp rx=0 tx=0).stop 2023-11-28T11:08:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 msgr2=0x7fd6c814e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 0x7fd6c814e020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c550 tx=0x7fd6bc00c860 comp rx=0 tx=0).stop 2023-11-28T11:08:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 shutdown_connections 2023-11-28T11:08:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6c80af1f0 0x7fd6c81548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd6b40682a0 0x7fd6b406a750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c806c370 0x7fd6c814e020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 --2- 172.21.15.83:0/3083348866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd6c80aee20 0x7fd6c8154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.576+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 >> 172.21.15.83:0/3083348866 conn(0x7fd6c80b2b60 msgr2=0x7fd6c806ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.577+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 shutdown_connections 2023-11-28T11:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.577+0000 7fd6cd889700 1 -- 172.21.15.83:0/3083348866 wait complete. 2023-11-28T11:08:31.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.714+0000 7f853bcf3700 1 Processor -- start 2023-11-28T11:08:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.714+0000 7f853bcf3700 1 -- start start 2023-11-28T11:08:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.715+0000 7f853bcf3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 0x7f85340b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.715+0000 7f853bcf3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85340d0e40 0x7f85340d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.715+0000 7f853bcf3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340d27c0 con 0x7f8534160bb0 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.715+0000 7f853bcf3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340cd8c0 con 0x7f85340d0e40 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.715+0000 7f853bcf3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340d2a60 con 0x7f85340b4980 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f853a290700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85340d0e40 0x7f85340d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 0x7f85340b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 0x7f85340b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:60450/0 (socket says 172.21.15.83:60450) 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 -- 172.21.15.83:0/996841636 learned_addr learned my addr 172.21.15.83:0/996841636 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 -- 172.21.15.83:0/996841636 >> v2:172.21.15.83:3300/0 conn(0x7f85340d0e40 msgr2=0x7f85340d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 --2- 172.21.15.83:0/996841636 >> v2:172.21.15.83:3300/0 conn(0x7f85340d0e40 0x7f85340d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 -- 172.21.15.83:0/996841636 >> v1:172.21.15.83:6789/0 conn(0x7f8534160bb0 legacy=0x7f85340b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 -- 172.21.15.83:0/996841636 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340cdba0 con 0x7f85340b4980 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8539a8f700 1 --2- 172.21.15.83:0/996841636 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 0x7f85340b5d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85300098f0 tx=0x7f8530009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa4df6671fb5a4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8538a8d700 1 -- 172.21.15.83:0/996841636 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530011830 con 0x7f85340b4980 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8538a8d700 1 -- 172.21.15.83:0/996841636 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530011990 con 0x7f85340b4980 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.716+0000 7f8538a8d700 1 -- 172.21.15.83:0/996841636 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530011d20 con 0x7f85340b4980 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 -- 172.21.15.83:0/996841636 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 msgr2=0x7f85340b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 --2- 172.21.15.83:0/996841636 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 0x7f85340b5d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85300098f0 tx=0x7f8530009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 -- 172.21.15.83:0/996841636 shutdown_connections 2023-11-28T11:08:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 --2- 172.21.15.83:0/996841636 >> v2:172.21.15.83:3300/0 conn(0x7f85340d0e40 0x7f85340d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 --2- 172.21.15.83:0/996841636 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f85340b4980 0x7f85340b5d50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 -- 172.21.15.83:0/996841636 >> 172.21.15.83:0/996841636 conn(0x7f853406ceb0 msgr2=0x7f85340cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 -- 172.21.15.83:0/996841636 shutdown_connections 2023-11-28T11:08:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.717+0000 7f853bcf3700 1 -- 172.21.15.83:0/996841636 wait complete. 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.719+0000 7f853bcf3700 1 Processor -- start 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.719+0000 7f853bcf3700 1 -- start start 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853bcf3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 0x7f8534159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853bcf3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340d0e40 0x7f85340cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853bcf3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8534160bb0 0x7f85340cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853bcf3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341f3020 con 0x7f85340d0e40 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853bcf3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853415aa40 con 0x7f85340b4980 2023-11-28T11:08:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853bcf3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853415f9f0 con 0x7f8534160bb0 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 0x7f8534159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 0x7f8534159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35016/0 (socket says 172.21.15.83:35016) 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 -- 172.21.15.83:0/457680736 learned_addr learned my addr 172.21.15.83:0/457680736 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853a290700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8534160bb0 0x7f85340cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f853928e700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340d0e40 0x7f85340cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 -- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8534160bb0 msgr2=0x7f85340cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8534160bb0 0x7f85340cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 -- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340d0e40 msgr2=0x7f85340cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340d0e40 0x7f85340cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.720+0000 7f8539a8f700 1 -- 172.21.15.83:0/457680736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009550 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.721+0000 7f8539a8f700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 0x7f8534159fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8530014040 tx=0x7f8530004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.721+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530004510 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.721+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853001e070 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.721+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85300038e0 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.721+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340cd550 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.721+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534161020 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.722+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85340cf380 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.722+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f853000f050 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.723+0000 7f852a7fc700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f85180682a0 0x7f851806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.723+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8530086810 con 0x7f85340b4980 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.723+0000 7f853928e700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f85180682a0 0x7f851806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.723+0000 7f853928e700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f85180682a0 0x7f851806a750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f85340b4760 tx=0x7f8524009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.727+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8530053330 con 0x7f85340b4980 2023-11-28T11:08:31.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.912+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85340742b0 con 0x7f85340b4980 2023-11-28T11:08:31.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.913+0000 7f852a7fc700 1 -- 172.21.15.83:0/457680736 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85300200a0 con 0x7f85340b4980 2023-11-28T11:08:31.914 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:08:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f85180682a0 msgr2=0x7f851806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f85180682a0 0x7f851806a750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f85340b4760 tx=0x7f8524009040 comp rx=0 tx=0).stop 2023-11-28T11:08:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 msgr2=0x7f8534159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 0x7f8534159fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8530014040 tx=0x7f8530004750 comp rx=0 tx=0).stop 2023-11-28T11:08:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 shutdown_connections 2023-11-28T11:08:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8534160bb0 0x7f85340cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85340b4980 0x7f8534159fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f85180682a0 0x7f851806a750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 --2- 172.21.15.83:0/457680736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340d0e40 0x7f85340cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 >> 172.21.15.83:0/457680736 conn(0x7f853406ceb0 msgr2=0x7f853415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 shutdown_connections 2023-11-28T11:08:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:31.915+0000 7f853bcf3700 1 -- 172.21.15.83:0/457680736 wait complete. 2023-11-28T11:08:31.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 Processor -- start 2023-11-28T11:08:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 -- start start 2023-11-28T11:08:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c00cf0d0 0x7f36c006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36c00b18a0 0x7f36c00b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c00cf540 con 0x7f36c0159830 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c015d480 con 0x7f36c00b18a0 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.050+0000 7f36c5e47700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b2260 con 0x7f36c00cf0d0 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36c00b18a0 0x7f36c00b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36c00b18a0 0x7f36c00b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48040/0 (socket says 172.21.15.83:48040) 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 -- 172.21.15.83:0/2228748288 learned_addr learned my addr 172.21.15.83:0/2228748288 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bf7fe700 1 --1- 172.21.15.83:0/2228748288 >> v1:172.21.15.83:6789/0 conn(0x7f36c0159830 0x7f36c015de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45896/0 (socket says 172.21.15.83:45896) 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36beffd700 1 --2- 172.21.15.83:0/2228748288 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c00cf0d0 0x7f36c006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 -- 172.21.15.83:0/2228748288 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c00cf0d0 msgr2=0x7f36c006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 --2- 172.21.15.83:0/2228748288 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c00cf0d0 0x7f36c006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 -- 172.21.15.83:0/2228748288 >> v1:172.21.15.83:6789/0 conn(0x7f36c0159830 legacy=0x7f36c015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 -- 172.21.15.83:0/2228748288 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00cf790 con 0x7f36c00b18a0 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36be7fc700 1 -- 172.21.15.83:0/2228748288 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652440574 0 0) 0x7f36c00cf540 con 0x7f36c0159830 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36bffff700 1 --2- 172.21.15.83:0/2228748288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00b18a0 0x7f36c00b1d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36b800bef0 tx=0x7f36b8010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604e235cd2b03944 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36be7fc700 1 -- 172.21.15.83:0/2228748288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800b2d0 con 0x7f36c00b18a0 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36be7fc700 1 -- 172.21.15.83:0/2228748288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b800b430 con 0x7f36c00b18a0 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.051+0000 7f36be7fc700 1 -- 172.21.15.83:0/2228748288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8009820 con 0x7f36c00b18a0 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 -- 172.21.15.83:0/2228748288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00b18a0 msgr2=0x7f36c00b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 --2- 172.21.15.83:0/2228748288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00b18a0 0x7f36c00b1d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36b800bef0 tx=0x7f36b8010730 comp rx=0 tx=0).stop 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 -- 172.21.15.83:0/2228748288 shutdown_connections 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 --2- 172.21.15.83:0/2228748288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00b18a0 0x7f36c00b1d20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 --2- 172.21.15.83:0/2228748288 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c00cf0d0 0x7f36c006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 -- 172.21.15.83:0/2228748288 >> 172.21.15.83:0/2228748288 conn(0x7f36c006e9a0 msgr2=0x7f36c00cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 -- 172.21.15.83:0/2228748288 shutdown_connections 2023-11-28T11:08:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.052+0000 7f36c5e47700 1 -- 172.21.15.83:0/2228748288 wait complete. 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.053+0000 7f36c5e47700 1 Processor -- start 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.053+0000 7f36c5e47700 1 -- start start 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36c5e47700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36c00b18a0 0x7f36c015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36c5e47700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 0x7f36c014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36c5e47700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c0159830 0x7f36c0152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36c5e47700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015bbf0 con 0x7f36c00cf0d0 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36c5e47700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015a610 con 0x7f36c00b18a0 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36c5e47700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015a8f0 con 0x7f36c0159830 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 0x7f36c014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 0x7f36c014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48044/0 (socket says 172.21.15.83:48044) 2023-11-28T11:08:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 -- 172.21.15.83:0/3580701781 learned_addr learned my addr 172.21.15.83:0/3580701781 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36bf7fe700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36c00b18a0 0x7f36c015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36bffff700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c0159830 0x7f36c0152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 -- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c0159830 msgr2=0x7f36c0152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c0159830 0x7f36c0152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 -- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36c00b18a0 msgr2=0x7f36c015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36c00b18a0 0x7f36c015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.054+0000 7f36beffd700 1 -- 172.21.15.83:0/3580701781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b800bb50 con 0x7f36c00cf0d0 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.055+0000 7f36beffd700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 0x7f36c014f320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f36b400e9b0 tx=0x7f36b400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.055+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4009710 con 0x7f36c00cf0d0 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.055+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b400ee50 con 0x7f36c00cf0d0 2023-11-28T11:08:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.055+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c015d690 con 0x7f36c00cf0d0 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.055+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400ca50 con 0x7f36c00cf0d0 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.055+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c00cfa70 con 0x7f36c00cf0d0 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.056+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c015d970 con 0x7f36c00cf0d0 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.056+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f36b4019030 con 0x7f36c00cf0d0 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.057+0000 7f36a7fff700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36ac0682a0 0x7f36ac06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.057+0000 7f36bf7fe700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36ac0682a0 0x7f36ac06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.057+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f36b4085750 con 0x7f36c00cf0d0 2023-11-28T11:08:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.057+0000 7f36bf7fe700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36ac0682a0 0x7f36ac06a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36b0000f60 tx=0x7f36b0007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.061+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f36b4052350 con 0x7f36c00cf0d0 2023-11-28T11:08:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.288+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c015b500 con 0x7f36c00cf0d0 2023-11-28T11:08:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.288+0000 7f36a7fff700 1 -- 172.21.15.83:0/3580701781 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f36b4017de0 con 0x7f36c00cf0d0 2023-11-28T11:08:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:32.289 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36ac0682a0 msgr2=0x7f36ac06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36ac0682a0 0x7f36ac06a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36b0000f60 tx=0x7f36b0007e70 comp rx=0 tx=0).stop 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 msgr2=0x7f36c014f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 0x7f36c014f320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f36b400e9b0 tx=0x7f36b400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 shutdown_connections 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c0159830 0x7f36c0152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36c00b18a0 0x7f36c015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36ac0682a0 0x7f36ac06a750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 --2- 172.21.15.83:0/3580701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c00cf0d0 0x7f36c014f320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.290+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 >> 172.21.15.83:0/3580701781 conn(0x7f36c006e9a0 msgr2=0x7f36c00cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.291+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 shutdown_connections 2023-11-28T11:08:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.291+0000 7f36c5e47700 1 -- 172.21.15.83:0/3580701781 wait complete. 2023-11-28T11:08:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2023-11-28T11:08:32.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:32.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.424+0000 7fabcb276700 1 Processor -- start 2023-11-28T11:08:32.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.425+0000 7fabcb276700 1 -- start start 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabcb276700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fabc415cd10 0x7fabc40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabcb276700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40cbfb0 con 0x7fabc40b1fb0 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabcb276700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40cb710 con 0x7fabc415d0e0 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabcb276700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc40cb9f0 con 0x7fabc415cd10 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabc9813700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fabc40b1fb0 0x7fabc406bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45914/0 (socket says 172.21.15.83:45914) 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabc9813700 1 -- 172.21.15.83:0/2391419850 learned_addr learned my addr 172.21.15.83:0/2391419850 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabc9012700 1 --2- 172.21.15.83:0/2391419850 >> v2:172.21.15.90:3300/0 conn(0x7fabc415cd10 0x7fabc40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915025756 0 0) 0x7fabc40cbfb0 con 0x7fabc40b1fb0 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb00035b0 con 0x7fabc40b1fb0 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1254990229 0 0) 0x7fabb00035b0 con 0x7fabc40b1fb0 2023-11-28T11:08:32.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 >> v2:172.21.15.90:3300/0 conn(0x7fabc415cd10 msgr2=0x7fabc40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 --2- 172.21.15.83:0/2391419850 >> v2:172.21.15.90:3300/0 conn(0x7fabc415cd10 0x7fabc40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 >> v1:172.21.15.90:6789/0 conn(0x7fabc415d0e0 legacy=0x7fabc40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40cbcd0 con 0x7fabc40b1fb0 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fabbc0019f0 con 0x7fabc40b1fb0 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 >> v1:172.21.15.83:6789/0 conn(0x7fabc40b1fb0 legacy=0x7fabc406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb0003a00 0x7fabb0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabb0004850 0x7fabb0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0005200 0x7fabb000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40cb9f0 con 0x7fabb0003a00 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb000c080 con 0x7fabb0005200 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb000c240 con 0x7fabb0004850 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabc9012700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb0003a00 0x7fabb0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabc9813700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0005200 0x7fabb000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.426+0000 7fabc8811700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabb0004850 0x7fabb0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9012700 1 -- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabb0004850 msgr2=0x7fabb0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9012700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabb0004850 0x7fabb0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9012700 1 -- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0005200 msgr2=0x7fabb000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9012700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0005200 0x7fabb000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9813700 1 -- 172.21.15.83:0/2391419850 reap_dead start 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9012700 1 -- 172.21.15.83:0/2391419850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00035b0 con 0x7fabb0003a00 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabc9012700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb0003a00 0x7fabb0003e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009b60 tx=0x7fabb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e70d1dde81e0ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400e6c0 con 0x7fabb0003a00 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb400e820 con 0x7fabb0003a00 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabbbfff700 1 -- 172.21.15.83:0/2391419850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400eb90 con 0x7fabb0003a00 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabcb276700 1 -- 172.21.15.83:0/2391419850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb0003a00 msgr2=0x7fabb0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabcb276700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb0003a00 0x7fabb0003e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009b60 tx=0x7fabb0005880 comp rx=0 tx=0).stop 2023-11-28T11:08:32.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabcb276700 1 -- 172.21.15.83:0/2391419850 shutdown_connections 2023-11-28T11:08:32.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabcb276700 1 --2- 172.21.15.83:0/2391419850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb0003a00 0x7fabb0003e10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.427+0000 7fabcb276700 1 -- 172.21.15.83:0/2391419850 >> 172.21.15.83:0/2391419850 conn(0x7fabc406e9a0 msgr2=0x7fabc40cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.428+0000 7fabcb276700 1 -- 172.21.15.83:0/2391419850 shutdown_connections 2023-11-28T11:08:32.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.428+0000 7fabcb276700 1 -- 172.21.15.83:0/2391419850 wait complete. 2023-11-28T11:08:32.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 Processor -- start 2023-11-28T11:08:32.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 -- start start 2023-11-28T11:08:32.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 0x7fabc40af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc40b4580 0x7fabc40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 0x7fabc40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40d0720 con 0x7fabc40cfd60 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40af580 con 0x7fabb0003a00 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.429+0000 7fabcb276700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40b4a40 con 0x7fabc40b4580 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 0x7fabc40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9012700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 0x7fabc40af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 0x7fabc40d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48066/0 (socket says 172.21.15.83:48066) 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 -- 172.21.15.83:0/4141998171 learned_addr learned my addr 172.21.15.83:0/4141998171 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9012700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 0x7fabc40af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35056/0 (socket says 172.21.15.83:35056) 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc8811700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc40b4580 0x7fabc40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 -- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc40b4580 msgr2=0x7fabc40c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc40b4580 0x7fabc40c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 -- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 msgr2=0x7fabc40af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 0x7fabc40af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 -- 172.21.15.83:0/4141998171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc406b660 con 0x7fabc40cfd60 2023-11-28T11:08:32.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9012700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 0x7fabc40af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabc9813700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 0x7fabc40d01e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fabc4072970 tx=0x7fabbc00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc00ac90 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc00ae30 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc406b9b0 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc013630 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.430+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc40cd9f0 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.431+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4074320 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.432+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fabbc013790 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.432+0000 7fabb9ffb700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faba80682a0 0x7faba806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.432+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fabbc0861c0 con 0x7fabc40cfd60 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.432+0000 7fabc9012700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faba80682a0 0x7faba806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.432+0000 7fabc9012700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faba80682a0 0x7faba806a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fabb40090b0 tx=0x7fabb401e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.436+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabbc052dc0 con 0x7fabc40cfd60 2023-11-28T11:08:32.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.443+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabbc053190 con 0x7fabc40cfd60 2023-11-28T11:08:32.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.619+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabc40b27b0 con 0x7fabc40cfd60 2023-11-28T11:08:32.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.620+0000 7fabb9ffb700 1 -- 172.21.15.83:0/4141998171 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14159 (secure 0 0 0) 0x7fabbc0567f0 con 0x7fabc40cfd60 2023-11-28T11:08:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:32.621 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:31.833679+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faba80682a0 msgr2=0x7faba806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faba80682a0 0x7faba806a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fabb40090b0 tx=0x7fabb401e040 comp rx=0 tx=0).stop 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 msgr2=0x7fabc40d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 0x7fabc40d01e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fabc4072970 tx=0x7fabbc00a5f0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 shutdown_connections 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faba80682a0 0x7faba806a750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabc40cfd60 0x7fabc40d01e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb0003a00 0x7fabc40af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 --2- 172.21.15.83:0/4141998171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc40b4580 0x7fabc40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 >> 172.21.15.83:0/4141998171 conn(0x7fabc406e9a0 msgr2=0x7fabc415a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 shutdown_connections 2023-11-28T11:08:32.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.622+0000 7fabcb276700 1 -- 172.21.15.83:0/4141998171 wait complete. 2023-11-28T11:08:32.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:32.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.756+0000 7f10a34e8700 1 Processor -- start 2023-11-28T11:08:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.756+0000 7f10a34e8700 1 -- start start 2023-11-28T11:08:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a34e8700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f109c0aee20 0x7f109c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a34e8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f109c0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a34e8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0b5b00 con 0x7f109c06c370 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a34e8700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c06c800 con 0x7f109c0aee20 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a34e8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c06cae0 con 0x7f109c0af1f0 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a0a83700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f109c06c370 0x7f109c06ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45930/0 (socket says 172.21.15.83:45930) 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a0a83700 1 -- 172.21.15.83:0/2229598478 learned_addr learned my addr 172.21.15.83:0/2229598478 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828104253 0 0) 0x7f1098001110 con 0x7f109c06c370 2023-11-28T11:08:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a1284700 1 --2- 172.21.15.83:0/2229598478 >> v2:172.21.15.90:3300/0 conn(0x7f109c0aee20 0x7f109c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10880035b0 con 0x7f109c06c370 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a1a85700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f109c0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156238689 0 0) 0x7f10880035b0 con 0x7f109c06c370 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 msgr2=0x7f109c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f109c0cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 >> v2:172.21.15.90:3300/0 conn(0x7f109c0aee20 msgr2=0x7f109c0b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 --2- 172.21.15.83:0/2229598478 >> v2:172.21.15.90:3300/0 conn(0x7f109c0aee20 0x7f109c0b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c06cdc0 con 0x7f109c06c370 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f1098001800 con 0x7f109c06c370 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a1a85700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f109c0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.757+0000 7f10a1284700 1 --2- 172.21.15.83:0/2229598478 >> v2:172.21.15.90:3300/0 conn(0x7f109c0aee20 0x7f109c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 >> v1:172.21.15.83:6789/0 conn(0x7f109c06c370 legacy=0x7f109c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1088003a00 0x7f1088003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f1088004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f1088005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c06c800 con 0x7f1088004850 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c06cae0 con 0x7f1088003a00 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108800bb10 con 0x7f109c0af1f0 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f1088004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a1284700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1088003a00 0x7f1088003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a1a85700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f1088005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 -- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 msgr2=0x7f1088005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c0af1f0 0x7f1088005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 -- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1088003a00 msgr2=0x7f1088003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1088003a00 0x7f1088003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 -- 172.21.15.83:0/2229598478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880035b0 con 0x7f1088004850 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f10a0a83700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f1088004cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f109c070d10 tx=0x7f1088005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916f28e8ec136f13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109800ce00 con 0x7f1088004850 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098005960 con 0x7f1088004850 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.758+0000 7f1093fff700 1 -- 172.21.15.83:0/2229598478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098005d10 con 0x7f1088004850 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a34e8700 1 -- 172.21.15.83:0/2229598478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 msgr2=0x7f1088004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a34e8700 1 --2- 172.21.15.83:0/2229598478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f1088004cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f109c070d10 tx=0x7f1088005880 comp rx=0 tx=0).stop 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a1a85700 1 -- 172.21.15.83:0/2229598478 reap_dead start 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a34e8700 1 -- 172.21.15.83:0/2229598478 shutdown_connections 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a34e8700 1 -- 172.21.15.83:0/2229598478 >> 172.21.15.83:0/2229598478 conn(0x7f109c0b2b60 msgr2=0x7f109c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a34e8700 1 -- 172.21.15.83:0/2229598478 shutdown_connections 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.759+0000 7f10a34e8700 1 -- 172.21.15.83:0/2229598478 wait complete. 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.760+0000 7f10a34e8700 1 Processor -- start 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.760+0000 7f10a34e8700 1 -- start start 2023-11-28T11:08:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f109c0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c14d930 0x7f109c1660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109c1665e0 0x7f109c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c14e9e0 con 0x7f1088004850 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c14ddd0 con 0x7f109c1665e0 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c14e0b0 con 0x7f109c14d930 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f109c0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f109c0c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48078/0 (socket says 172.21.15.83:48078) 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 -- 172.21.15.83:0/3819713018 learned_addr learned my addr 172.21.15.83:0/3819713018 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a0a83700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c14d930 0x7f109c1660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1a85700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109c1665e0 0x7f109c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 -- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c14d930 msgr2=0x7f109c1660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c14d930 0x7f109c1660a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 -- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109c1665e0 msgr2=0x7f109c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109c1665e0 0x7f109c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 -- 172.21.15.83:0/3819713018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c06fc70 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a0a83700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c14d930 0x7f109c1660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1a85700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109c1665e0 0x7f109c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a1284700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f109c0c5550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f108c011900 tx=0x7f108c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c004230 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c004390 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0036a0 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.761+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c06d0a0 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.762+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c06d380 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.762+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c0742b0 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.763+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f108c025070 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.763+0000 7f1091ffb700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10800682f0 0x7f108006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.763+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f108c086010 con 0x7f1088004850 2023-11-28T11:08:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.763+0000 7f10a0a83700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10800682f0 0x7f108006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.764+0000 7f10a0a83700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10800682f0 0x7f108006a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1098015040 tx=0x7f109800a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.768+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f108c052ad0 con 0x7f1088004850 2023-11-28T11:08:32.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.949+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f109c06bf40 con 0x7f1088004850 2023-11-28T11:08:32.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.949+0000 7f1091ffb700 1 -- 172.21.15.83:0/3819713018 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f108c00c0a0 con 0x7f1088004850 2023-11-28T11:08:32.950 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:08:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10800682f0 msgr2=0x7f108006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10800682f0 0x7f108006a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1098015040 tx=0x7f109800a000 comp rx=0 tx=0).stop 2023-11-28T11:08:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 msgr2=0x7f109c0c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f109c0c5550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f108c011900 tx=0x7f108c011cc0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 shutdown_connections 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109c1665e0 0x7f109c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10800682f0 0x7f108006a7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088004850 0x7f109c0c5550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.951+0000 7f10a34e8700 1 --2- 172.21.15.83:0/3819713018 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109c14d930 0x7f109c1660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.952+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 >> 172.21.15.83:0/3819713018 conn(0x7f109c0b2b60 msgr2=0x7f109c0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.952+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 shutdown_connections 2023-11-28T11:08:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:32.952+0000 7f10a34e8700 1 -- 172.21.15.83:0/3819713018 wait complete. 2023-11-28T11:08:32.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:33.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.086+0000 7f1c90f82700 1 Processor -- start 2023-11-28T11:08:33.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.086+0000 7f1c90f82700 1 -- start start 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c90f82700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c06e370 0x7f1c8c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c90f82700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c8c0b4bb0 0x7f1c8c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c90f82700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8c06d4a0 con 0x7f1c8c15c890 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c90f82700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8c0b5050 con 0x7f1c8c0b4bb0 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c90f82700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0b5ae0 con 0x7f1c8c06e370 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8a59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1c8c15c890 0x7f1c8c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45946/0 (socket says 172.21.15.83:45946) 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8a59c700 1 -- 172.21.15.83:0/3532323757 learned_addr learned my addr 172.21.15.83:0/3532323757 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8ad9d700 1 --2- 172.21.15.83:0/3532323757 >> v2:172.21.15.83:3300/0 conn(0x7f1c8c0b4bb0 0x7f1c8c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c89d9b700 1 --2- 172.21.15.83:0/3532323757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c06e370 0x7f1c8c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8959a700 1 -- 172.21.15.83:0/3532323757 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651083201 0 0) 0x7f1c8c06d4a0 con 0x7f1c8c15c890 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8959a700 1 -- 172.21.15.83:0/3532323757 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c740035b0 con 0x7f1c8c15c890 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8ad9d700 1 -- 172.21.15.83:0/3532323757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c06e370 msgr2=0x7f1c8c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8ad9d700 1 --2- 172.21.15.83:0/3532323757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c06e370 0x7f1c8c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8ad9d700 1 -- 172.21.15.83:0/3532323757 >> v1:172.21.15.83:6789/0 conn(0x7f1c8c15c890 legacy=0x7f1c8c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8ad9d700 1 -- 172.21.15.83:0/3532323757 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c06c400 con 0x7f1c8c0b4bb0 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8ad9d700 1 --2- 172.21.15.83:0/3532323757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c06b150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009d90 tx=0x7f1c8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0afd0c24acc69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.087+0000 7f1c8959a700 1 -- 172.21.15.83:0/3532323757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400e070 con 0x7f1c8c0b4bb0 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c8959a700 1 -- 172.21.15.83:0/3532323757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8400bb20 con 0x7f1c8c0b4bb0 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c8959a700 1 -- 172.21.15.83:0/3532323757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84012430 con 0x7f1c8c0b4bb0 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 -- 172.21.15.83:0/3532323757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c0b4bb0 msgr2=0x7f1c8c06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 --2- 172.21.15.83:0/3532323757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c06b150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009d90 tx=0x7f1c8400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:33.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 -- 172.21.15.83:0/3532323757 shutdown_connections 2023-11-28T11:08:33.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 --2- 172.21.15.83:0/3532323757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c06b150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 --2- 172.21.15.83:0/3532323757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c06e370 0x7f1c8c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 -- 172.21.15.83:0/3532323757 >> 172.21.15.83:0/3532323757 conn(0x7f1c8c0b2b60 msgr2=0x7f1c8c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 -- 172.21.15.83:0/3532323757 shutdown_connections 2023-11-28T11:08:33.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.088+0000 7f1c90f82700 1 -- 172.21.15.83:0/3532323757 wait complete. 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.089+0000 7f1c90f82700 1 Processor -- start 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 -- start start 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 0x7f1c8c0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c15c890 0x7f1c8c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c1592e0 con 0x7f1c8c06e370 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0caad0 con 0x7f1c8c0b4bb0 2023-11-28T11:08:33.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c90f82700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0cadb0 con 0x7f1c8c15c890 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c8a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 0x7f1c8c0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c8a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 0x7f1c8c0b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48094/0 (socket says 172.21.15.83:48094) 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.090+0000 7f1c8a59c700 1 -- 172.21.15.83:0/415531026 learned_addr learned my addr 172.21.15.83:0/415531026 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8ad9d700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c15c890 0x7f1c8c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c89d9b700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8a59c700 1 -- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c15c890 msgr2=0x7f1c8c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8a59c700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c15c890 0x7f1c8c0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8a59c700 1 -- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c8c0b4bb0 msgr2=0x7f1c8c14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8a59c700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8a59c700 1 -- 172.21.15.83:0/415531026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84009a70 con 0x7f1c8c06e370 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c89d9b700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8ad9d700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c15c890 0x7f1c8c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:33.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c8a59c700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 0x7f1c8c0b1220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c001260 tx=0x7f1c7c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0074e0 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c007640 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c005a50 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c0b5520 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.091+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c0b5240 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.092+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c0b5800 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.092+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c7c019030 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.093+0000 7f1c7affd700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c6c0682f0 0x7f1c6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.093+0000 7f1c89d9b700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c6c0682f0 0x7f1c6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.093+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1c7c084d30 con 0x7f1c8c06e370 2023-11-28T11:08:33.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.093+0000 7f1c89d9b700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c6c0682f0 0x7f1c6c06a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c159970 tx=0x7f1c80008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.097+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c7c0518e0 con 0x7f1c8c06e370 2023-11-28T11:08:33.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.277+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c8c0742b0 con 0x7f1c8c06e370 2023-11-28T11:08:33.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.277+0000 7f1c7affd700 1 -- 172.21.15.83:0/415531026 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c7c0170a0 con 0x7f1c8c06e370 2023-11-28T11:08:33.278 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c6c0682f0 msgr2=0x7f1c6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c6c0682f0 0x7f1c6c06a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c159970 tx=0x7f1c80008040 comp rx=0 tx=0).stop 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 msgr2=0x7f1c8c0b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 0x7f1c8c0b1220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c001260 tx=0x7f1c7c00c860 comp rx=0 tx=0).stop 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 shutdown_connections 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c8c15c890 0x7f1c8c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c6c0682f0 0x7f1c6c06a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8c06e370 0x7f1c8c0b1220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 --2- 172.21.15.83:0/415531026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c8c0b4bb0 0x7f1c8c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 >> 172.21.15.83:0/415531026 conn(0x7f1c8c0b2b60 msgr2=0x7f1c8c0ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 shutdown_connections 2023-11-28T11:08:33.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.279+0000 7f1c90f82700 1 -- 172.21.15.83:0/415531026 wait complete. 2023-11-28T11:08:33.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:08:33.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.411+0000 7fec9d6e7700 1 Processor -- start 2023-11-28T11:08:33.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.411+0000 7fec9d6e7700 1 -- start start 2023-11-28T11:08:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.412+0000 7fec9d6e7700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fec9815d0e0 0x7fec980ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.412+0000 7fec9d6e7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980cbfb0 con 0x7fec980b1fb0 2023-11-28T11:08:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.412+0000 7fec9d6e7700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980cb710 con 0x7fec9815cd10 2023-11-28T11:08:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.412+0000 7fec9d6e7700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec980cb9f0 con 0x7fec9815d0e0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.412+0000 7fec977fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fec980b1fb0 0x7fec9806bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45964/0 (socket says 172.21.15.83:45964) 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.412+0000 7fec977fe700 1 -- 172.21.15.83:0/1941510674 learned_addr learned my addr 172.21.15.83:0/1941510674 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922980999 0 0) 0x7fec980cbfb0 con 0x7fec980b1fb0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec967fc700 1 --2- 172.21.15.83:0/1941510674 >> v2:172.21.15.90:3300/0 conn(0x7fec9815d0e0 0x7fec980ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec800035b0 con 0x7fec980b1fb0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318728657 0 0) 0x7fec800035b0 con 0x7fec980b1fb0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 >> v2:172.21.15.90:3300/0 conn(0x7fec9815d0e0 msgr2=0x7fec980ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 --2- 172.21.15.83:0/1941510674 >> v2:172.21.15.90:3300/0 conn(0x7fec9815d0e0 0x7fec980ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 >> v1:172.21.15.90:6789/0 conn(0x7fec9815cd10 legacy=0x7fec980b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980cbcd0 con 0x7fec980b1fb0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fec900019f0 con 0x7fec980b1fb0 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 >> v1:172.21.15.83:6789/0 conn(0x7fec980b1fb0 legacy=0x7fec9806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec80003a00 0x7fec80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec80004850 0x7fec80004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec8000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980cb9f0 con 0x7fec80005200 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8000c080 con 0x7fec80004850 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8000c240 con 0x7fec80003a00 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec8000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec967fc700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec80004850 0x7fec80004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec96ffd700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec80003a00 0x7fec80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 -- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec80003a00 msgr2=0x7fec80003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec80003a00 0x7fec80003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 -- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec80004850 msgr2=0x7fec80004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec80004850 0x7fec80004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 -- 172.21.15.83:0/1941510674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec800035b0 con 0x7fec80005200 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 -- 172.21.15.83:0/1941510674 reap_dead start 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.413+0000 7fec977fe700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec8000ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fec9000a800 tx=0x7fec80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286bac69c85a7fc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000a2f0 con 0x7fec80005200 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec90012620 con 0x7fec80005200 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec95ffb700 1 -- 172.21.15.83:0/1941510674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900129d0 con 0x7fec80005200 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 -- 172.21.15.83:0/1941510674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 msgr2=0x7fec8000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec8000ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fec9000a800 tx=0x7fec80005880 comp rx=0 tx=0).stop 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 -- 172.21.15.83:0/1941510674 shutdown_connections 2023-11-28T11:08:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/1941510674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec8000ba10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 -- 172.21.15.83:0/1941510674 >> 172.21.15.83:0/1941510674 conn(0x7fec9806e9a0 msgr2=0x7fec980cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 -- 172.21.15.83:0/1941510674 shutdown_connections 2023-11-28T11:08:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.414+0000 7fec9d6e7700 1 -- 172.21.15.83:0/1941510674 wait complete. 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.416+0000 7fec9d6e7700 1 Processor -- start 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.416+0000 7fec9d6e7700 1 -- start start 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec981575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec981f3a60 0x7fec980b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec981f3ed0 0x7fec9815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980c7070 con 0x7fec80005200 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980c7330 con 0x7fec981f3a60 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980c7610 con 0x7fec981f3ed0 2023-11-28T11:08:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec981575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec981575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48112/0 (socket says 172.21.15.83:48112) 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 -- 172.21.15.83:0/274199754 learned_addr learned my addr 172.21.15.83:0/274199754 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec967fc700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec981f3a60 0x7fec980b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec977fe700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec981f3ed0 0x7fec9815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 -- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec981f3ed0 msgr2=0x7fec9815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec981f3ed0 0x7fec9815fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 -- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec981f3a60 msgr2=0x7fec980b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec981f3a60 0x7fec980b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 -- 172.21.15.83:0/274199754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9806b660 con 0x7fec80005200 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec967fc700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec981f3a60 0x7fec980b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec96ffd700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec981575c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00f7f0 tx=0x7fec8c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0052e0 con 0x7fec80005200 2023-11-28T11:08:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c005440 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9806b9b0 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.417+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0038a0 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.418+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec980cd9f0 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.418+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec980c7c60 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.419+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec8c003a00 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.419+0000 7fec7f7fe700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fec840682f0 0x7fec8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.419+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fec8c0858b0 con 0x7fec80005200 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.419+0000 7fec967fc700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fec840682f0 0x7fec8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.420+0000 7fec967fc700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fec840682f0 0x7fec8406a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fec980c9850 tx=0x7fec88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.423+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fec8c052460 con 0x7fec80005200 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.610+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fec98074320 con 0x7fec80005200 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.611+0000 7fec7f7fe700 1 -- 172.21.15.83:0/274199754 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7fec8c055a80 con 0x7fec80005200 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": "rbd" 2023-11-28T11:08:33.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:08:33.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:08:33.612 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 2, 2023-11-28T11:08:33.612 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-28T11:08:33.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:08:33.612 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.612+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fec840682f0 msgr2=0x7fec8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.612+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fec840682f0 0x7fec8406a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fec980c9850 tx=0x7fec88008040 comp rx=0 tx=0).stop 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.612+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 msgr2=0x7fec981575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.612+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec981575c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00f7f0 tx=0x7fec8c00fbb0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 shutdown_connections 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec981f3ed0 0x7fec9815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fec840682f0 0x7fec8406a7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec80005200 0x7fec981575c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 --2- 172.21.15.83:0/274199754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec981f3a60 0x7fec980b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 >> 172.21.15.83:0/274199754 conn(0x7fec9806e9a0 msgr2=0x7fec980af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 shutdown_connections 2023-11-28T11:08:33.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.613+0000 7fec9d6e7700 1 -- 172.21.15.83:0/274199754 wait complete. 2023-11-28T11:08:33.625 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:08:33.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:08:33.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.748+0000 7f83650df700 1 Processor -- start 2023-11-28T11:08:33.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.748+0000 7f83650df700 1 -- start start 2023-11-28T11:08:33.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f83650df700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f836015e000 0x7f83601595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f83650df700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 0x7f8360159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f83650df700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836006bc10 con 0x7f83600b1900 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f83650df700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83600b57d0 con 0x7f836015e000 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f83650df700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836006cc20 con 0x7f83600b1cd0 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835ed9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f83600b1900 0x7f83600b50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45980/0 (socket says 172.21.15.83:45980) 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835ed9d700 1 -- 172.21.15.83:0/1500978540 learned_addr learned my addr 172.21.15.83:0/1500978540 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835e59c700 1 --2- 172.21.15.83:0/1500978540 >> v2:172.21.15.90:3300/0 conn(0x7f836015e000 0x7f83601595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835f59e700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 0x7f8360159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052073985 0 0) 0x7f836006bc10 con 0x7f83600b1900 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83480035b0 con 0x7f83600b1900 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3205232277 0 0) 0x7f83480035b0 con 0x7f83600b1900 2023-11-28T11:08:33.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 msgr2=0x7f8360159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 0x7f8360159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 >> v2:172.21.15.90:3300/0 conn(0x7f836015e000 msgr2=0x7f83601595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 --2- 172.21.15.83:0/1500978540 >> v2:172.21.15.90:3300/0 conn(0x7f836015e000 0x7f83601595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836006be60 con 0x7f83600b1900 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f8350001670 con 0x7f83600b1900 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 >> v1:172.21.15.83:6789/0 conn(0x7f83600b1900 legacy=0x7f83600b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 0x7f8348003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348003f80 0x7f8348004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348004d90 0x7f8348005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600b57d0 con 0x7f8348003f80 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836006cc20 con 0x7f8348004d90 2023-11-28T11:08:33.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834800ba40 con 0x7f83600b1cd0 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835e59c700 1 --2- 172.21.15.83:0/1500978540 >> v2:172.21.15.90:3300/0 conn(0x7f836015e000 0x7f83601595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835ed9d700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 0x7f8348003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835e59c700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348003f80 0x7f8348004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.749+0000 7f835f59e700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348004d90 0x7f8348005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835e59c700 1 -- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 msgr2=0x7f8348003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835e59c700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600b1cd0 0x7f8348003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835e59c700 1 -- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348004d90 msgr2=0x7f8348005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835e59c700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348004d90 0x7f8348005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835e59c700 1 -- 172.21.15.83:0/1500978540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83480035b0 con 0x7f8348003f80 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835f59e700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348004d90 0x7f8348005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835e59c700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348003f80 0x7f8348004850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8354009d10 tx=0x7f8348005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c2146d4e9067c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354009e60 con 0x7f8348003f80 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8354004030 con 0x7f8348003f80 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835dd9b700 1 -- 172.21.15.83:0/1500978540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540043a0 con 0x7f8348003f80 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f83650df700 1 -- 172.21.15.83:0/1500978540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348003f80 msgr2=0x7f8348004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f83650df700 1 --2- 172.21.15.83:0/1500978540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348003f80 0x7f8348004850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8354009d10 tx=0x7f8348005880 comp rx=0 tx=0).stop 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f835f59e700 1 -- 172.21.15.83:0/1500978540 reap_dead start 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f83650df700 1 -- 172.21.15.83:0/1500978540 shutdown_connections 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f83650df700 1 -- 172.21.15.83:0/1500978540 >> 172.21.15.83:0/1500978540 conn(0x7f83600cc690 msgr2=0x7f83600cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f83650df700 1 -- 172.21.15.83:0/1500978540 shutdown_connections 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.750+0000 7f83650df700 1 -- 172.21.15.83:0/1500978540 wait complete. 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.752+0000 7f83650df700 1 Processor -- start 2023-11-28T11:08:33.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.752+0000 7f83650df700 1 -- start start 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348003f80 0x7f83600ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 0x7f83600cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600c95d0 0x7f8360148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cfbc0 con 0x7f83600c9160 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cd8f0 con 0x7f8348003f80 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cdbd0 con 0x7f83600c95d0 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 0x7f83600cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 0x7f83600cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48134/0 (socket says 172.21.15.83:48134) 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 -- 172.21.15.83:0/2175126123 learned_addr learned my addr 172.21.15.83:0/2175126123 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835ed9d700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348003f80 0x7f83600ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835f59e700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600c95d0 0x7f8360148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 -- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600c95d0 msgr2=0x7f8360148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600c95d0 0x7f8360148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 -- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348003f80 msgr2=0x7f83600ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348003f80 0x7f83600ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 -- 172.21.15.83:0/2175126123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836006c420 con 0x7f83600c9160 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f835e59c700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 0x7f83600cf170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8348004770 tx=0x7f83540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354020070 con 0x7f83600c9160 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8354009e60 con 0x7f83600c9160 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836006c140 con 0x7f83600c9160 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354003af0 con 0x7f83600c9160 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.753+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836006c700 con 0x7f83600c9160 2023-11-28T11:08:33.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.754+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83600742b0 con 0x7f83600c9160 2023-11-28T11:08:33.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.755+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f835402c030 con 0x7f83600c9160 2023-11-28T11:08:33.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.755+0000 7f83477fe700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f834c0682f0 0x7f834c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.755+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8354086dc0 con 0x7f83600c9160 2023-11-28T11:08:33.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.755+0000 7f835ed9d700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f834c0682f0 0x7f834c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.755+0000 7f835ed9d700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f834c0682f0 0x7f834c06a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f8350009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.759+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8354053970 con 0x7f83600c9160 2023-11-28T11:08:33.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:33.942+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f83600cfbc0 con 0x7f83600c9160 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.118+0000 7f83477fe700 1 -- 172.21.15.83:0/2175126123 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8354056f90 con 0x7f83600c9160 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f834c0682f0 msgr2=0x7f834c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f834c0682f0 0x7f834c06a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f8350009040 comp rx=0 tx=0).stop 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 msgr2=0x7f83600cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 0x7f83600cf170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8348004770 tx=0x7f83540047c0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 shutdown_connections 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83600c95d0 0x7f8360148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8348003f80 0x7f83600ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f834c0682f0 0x7f834c06a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 --2- 172.21.15.83:0/2175126123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83600c9160 0x7f83600cf170 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 >> 172.21.15.83:0/2175126123 conn(0x7f83600cc690 msgr2=0x7f83600cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 shutdown_connections 2023-11-28T11:08:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.120+0000 7f83650df700 1 -- 172.21.15.83:0/2175126123 wait complete. 2023-11-28T11:08:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-28T11:08:34.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-28T11:08:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.262+0000 7fcf8b993700 1 Processor -- start 2023-11-28T11:08:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.262+0000 7fcf8b993700 1 -- start start 2023-11-28T11:08:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8b993700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf8415c680 0x7fcf840cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8b993700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcf8415d650 0x7fcf8406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8b993700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8415ca50 con 0x7fcf840d0cc0 2023-11-28T11:08:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8b993700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf840d0a00 con 0x7fcf8415c680 2023-11-28T11:08:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8b993700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf84070260 con 0x7fcf8415d650 2023-11-28T11:08:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8972f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf8415c680 0x7fcf840cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf89f30700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcf840d0cc0 0x7fcf840cd850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46004/0 (socket says 172.21.15.83:46004) 2023-11-28T11:08:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf89f30700 1 -- 172.21.15.83:0/697703220 learned_addr learned my addr 172.21.15.83:0/697703220 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf88f2e700 1 --2- 172.21.15.83:0/697703220 >> v2:172.21.15.90:3300/0 conn(0x7fcf8415d650 0x7fcf8406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8972f700 1 -- 172.21.15.83:0/697703220 >> v2:172.21.15.90:3300/0 conn(0x7fcf8415d650 msgr2=0x7fcf8406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8972f700 1 --2- 172.21.15.83:0/697703220 >> v2:172.21.15.90:3300/0 conn(0x7fcf8415d650 0x7fcf8406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8972f700 1 -- 172.21.15.83:0/697703220 >> v1:172.21.15.83:6789/0 conn(0x7fcf840d0cc0 legacy=0x7fcf840cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf7bfff700 1 -- 172.21.15.83:0/697703220 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406055424 0 0) 0x7fcf8415ca50 con 0x7fcf840d0cc0 2023-11-28T11:08:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8972f700 1 -- 172.21.15.83:0/697703220 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8415dba0 con 0x7fcf8415c680 2023-11-28T11:08:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf88f2e700 1 --2- 172.21.15.83:0/697703220 >> v2:172.21.15.90:3300/0 conn(0x7fcf8415d650 0x7fcf8406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf8972f700 1 --2- 172.21.15.83:0/697703220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415c680 0x7fcf840cb380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406f590 tx=0x7fcf80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b1d7beeecf7638 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.263+0000 7fcf7bfff700 1 -- 172.21.15.83:0/697703220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80011830 con 0x7fcf8415c680 2023-11-28T11:08:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf7bfff700 1 -- 172.21.15.83:0/697703220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf80011990 con 0x7fcf8415c680 2023-11-28T11:08:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf7bfff700 1 -- 172.21.15.83:0/697703220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80011d20 con 0x7fcf8415c680 2023-11-28T11:08:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 -- 172.21.15.83:0/697703220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415c680 msgr2=0x7fcf840cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 --2- 172.21.15.83:0/697703220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415c680 0x7fcf840cb380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406f590 tx=0x7fcf80009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 -- 172.21.15.83:0/697703220 shutdown_connections 2023-11-28T11:08:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 --2- 172.21.15.83:0/697703220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415c680 0x7fcf840cb380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 --2- 172.21.15.83:0/697703220 >> v2:172.21.15.90:3300/0 conn(0x7fcf8415d650 0x7fcf8406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 -- 172.21.15.83:0/697703220 >> 172.21.15.83:0/697703220 conn(0x7fcf840708a0 msgr2=0x7fcf84070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 -- 172.21.15.83:0/697703220 shutdown_connections 2023-11-28T11:08:34.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.264+0000 7fcf8b993700 1 -- 172.21.15.83:0/697703220 wait complete. 2023-11-28T11:08:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.265+0000 7fcf8b993700 1 Processor -- start 2023-11-28T11:08:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.265+0000 7fcf8b993700 1 -- start start 2023-11-28T11:08:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8b993700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf840d0cc0 0x7fcf841580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8b993700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf8415c680 0x7fcf840b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8b993700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 0x7fcf840b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8b993700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840c9cc0 con 0x7fcf8415d650 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8b993700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840c9040 con 0x7fcf8415c680 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8b993700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840c9320 con 0x7fcf840d0cc0 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 0x7fcf840b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 0x7fcf840b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48150/0 (socket says 172.21.15.83:48150) 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 -- 172.21.15.83:0/1693417418 learned_addr learned my addr 172.21.15.83:0/1693417418 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf88f2e700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf8415c680 0x7fcf840b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf8972f700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf840d0cc0 0x7fcf841580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 -- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf840d0cc0 msgr2=0x7fcf841580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf840d0cc0 0x7fcf841580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 -- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf8415c680 msgr2=0x7fcf840b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf8415c680 0x7fcf840b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf89f30700 1 -- 172.21.15.83:0/1693417418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf800094a0 con 0x7fcf8415d650 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.266+0000 7fcf88f2e700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf8415c680 0x7fcf840b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.267+0000 7fcf89f30700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 0x7fcf840b5120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00cca0 tx=0x7fcf7c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.267+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00cea0 con 0x7fcf8415d650 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.267+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c00ade0 con 0x7fcf8415d650 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.267+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf840cd370 con 0x7fcf8415d650 2023-11-28T11:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.267+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c007310 con 0x7fcf8415d650 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.267+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf840cea30 con 0x7fcf8415d650 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.268+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf840cecd0 con 0x7fcf8415d650 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.268+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf7c01a030 con 0x7fcf8415d650 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.268+0000 7fcf79ffb700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcf680682f0 0x7fcf6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.269+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fcf7c085890 con 0x7fcf8415d650 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.269+0000 7fcf8972f700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcf680682f0 0x7fcf6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.269+0000 7fcf8972f700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcf680682f0 0x7fcf6806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf800097c0 tx=0x7fcf8001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.273+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf7c0522d0 con 0x7fcf8415d650 2023-11-28T11:08:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:34.453+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fcf840c9e00 con 0x7fcf8415d650 2023-11-28T11:08:35.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.124+0000 7fcf79ffb700 1 -- 172.21.15.83:0/1693417418 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fcf840c9e00 con 0x7fcf8415d650 2023-11-28T11:08:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.128+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcf680682f0 msgr2=0x7fcf6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.128+0000 7fcf8b993700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcf680682f0 0x7fcf6806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf800097c0 tx=0x7fcf8001c040 comp rx=0 tx=0).stop 2023-11-28T11:08:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.128+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 msgr2=0x7fcf840b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.128+0000 7fcf8b993700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 0x7fcf840b5120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00cca0 tx=0x7fcf7c00a340 comp rx=0 tx=0).stop 2023-11-28T11:08:35.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.134+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 shutdown_connections 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.134+0000 7fcf8b993700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcf680682f0 0x7fcf6806a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.134+0000 7fcf8b993700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf8415d650 0x7fcf840b5120 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.134+0000 7fcf8b993700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf840d0cc0 0x7fcf841580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.134+0000 7fcf8b993700 1 --2- 172.21.15.83:0/1693417418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf8415c680 0x7fcf840b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.134+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 >> 172.21.15.83:0/1693417418 conn(0x7fcf840708a0 msgr2=0x7fcf841609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.135+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 shutdown_connections 2023-11-28T11:08:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.135+0000 7fcf8b993700 1 -- 172.21.15.83:0/1693417418 wait complete. 2023-11-28T11:08:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-28T11:08:35.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:08:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.281+0000 7fbdbb563700 1 Processor -- start 2023-11-28T11:08:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.281+0000 7fbdbb563700 1 -- start start 2023-11-28T11:08:35.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdbb563700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdb40b5880 0x7fbdb40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdbb563700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b1bd0 0x7fbdb40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdbb563700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb40caf10 con 0x7fbdb415aa30 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdbb563700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb406c190 con 0x7fbdb40b5880 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdbb563700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40ca300 con 0x7fbdb40b1bd0 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdb8afe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdb40b5880 0x7fbdb40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdb8afe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdb40b5880 0x7fbdb40b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48156/0 (socket says 172.21.15.83:48156) 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdb8afe700 1 -- 172.21.15.83:0/2023941044 learned_addr learned my addr 172.21.15.83:0/2023941044 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.282+0000 7fbdb9b00700 1 --2- 172.21.15.83:0/2023941044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b1bd0 0x7fbdb40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdb8afe700 1 -- 172.21.15.83:0/2023941044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b1bd0 msgr2=0x7fbdb40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdb8afe700 1 --2- 172.21.15.83:0/2023941044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b1bd0 0x7fbdb40cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdb8afe700 1 -- 172.21.15.83:0/2023941044 >> v1:172.21.15.90:6789/0 conn(0x7fbdb415aa30 legacy=0x7fbdb40b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdb8afe700 1 -- 172.21.15.83:0/2023941044 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb40ca5e0 con 0x7fbdb40b5880 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdb8afe700 1 --2- 172.21.15.83:0/2023941044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40b5880 0x7fbdb40b1690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbda40098f0 tx=0x7fbda4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6f2351f5338d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdabfff700 1 -- 172.21.15.83:0/2023941044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4004230 con 0x7fbdb40b5880 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdabfff700 1 -- 172.21.15.83:0/2023941044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda4004390 con 0x7fbdb40b5880 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdabfff700 1 -- 172.21.15.83:0/2023941044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda40036a0 con 0x7fbdb40b5880 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdbb563700 1 -- 172.21.15.83:0/2023941044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40b5880 msgr2=0x7fbdb40b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2023941044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40b5880 0x7fbdb40b1690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbda40098f0 tx=0x7fbda4009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdbb563700 1 -- 172.21.15.83:0/2023941044 shutdown_connections 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2023941044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b1bd0 0x7fbdb40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2023941044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40b5880 0x7fbdb40b1690 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.283+0000 7fbdbb563700 1 -- 172.21.15.83:0/2023941044 >> 172.21.15.83:0/2023941044 conn(0x7fbdb40cc690 msgr2=0x7fbdb40cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.284+0000 7fbdbb563700 1 -- 172.21.15.83:0/2023941044 shutdown_connections 2023-11-28T11:08:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.284+0000 7fbdbb563700 1 -- 172.21.15.83:0/2023941044 wait complete. 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 Processor -- start 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 -- start start 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb40b1bd0 0x7fbdb41e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b5880 0x7fbdb40c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 0x7fbdb4148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb41e6e80 con 0x7fbdb415aa30 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb41e6270 con 0x7fbdb40b1bd0 2023-11-28T11:08:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdbb563700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb41e6550 con 0x7fbdb40b5880 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdb9b00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 0x7fbdb4148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdb9b00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 0x7fbdb4148980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48164/0 (socket says 172.21.15.83:48164) 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.285+0000 7fbdb9b00700 1 -- 172.21.15.83:0/2408918284 learned_addr learned my addr 172.21.15.83:0/2408918284 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb8afe700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b5880 0x7fbdb40c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb92ff700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb40b1bd0 0x7fbdb41e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb9b00700 1 -- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b5880 msgr2=0x7fbdb40c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb9b00700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b5880 0x7fbdb40c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb9b00700 1 -- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb40b1bd0 msgr2=0x7fbdb41e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb9b00700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb40b1bd0 0x7fbdb41e36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb9b00700 1 -- 172.21.15.83:0/2408918284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4009550 con 0x7fbdb415aa30 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb8afe700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b5880 0x7fbdb40c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdb9b00700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 0x7fbdb4148980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000e810 tx=0x7fbdb000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0013610 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0013770 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb40caba0 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0013b20 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.286+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb40ca8c0 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.287+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb40b2070 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.288+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbdb0023070 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.288+0000 7fbda9ffb700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd980682f0 0x7fbd9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.288+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fbdb00858d0 con 0x7fbdb415aa30 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.288+0000 7fbdb92ff700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd980682f0 0x7fbd9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.288+0000 7fbdb92ff700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd980682f0 0x7fbd9806a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009f20 tx=0x7fbdac0094d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.292+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbdb0052190 con 0x7fbdb415aa30 2023-11-28T11:08:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.517+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbdb4157ec0 con 0x7fbdb415aa30 2023-11-28T11:08:35.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.522+0000 7fbda9ffb700 1 -- 172.21.15.83:0/2408918284 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbdb4157ec0 con 0x7fbdb415aa30 2023-11-28T11:08:35.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.526+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd980682f0 msgr2=0x7fbd9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.526+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd980682f0 0x7fbd9806a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009f20 tx=0x7fbdac0094d0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.526+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 msgr2=0x7fbdb4148980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.526+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 0x7fbdb4148980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000e810 tx=0x7fbdb000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.527+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 shutdown_connections 2023-11-28T11:08:35.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.527+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd980682f0 0x7fbd9806a7a0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009f20 tx=0x7fbdac0094d0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.527+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb415aa30 0x7fbdb4148980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.527+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb40b1bd0 0x7fbdb41e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.527+0000 7fbdbb563700 1 --2- 172.21.15.83:0/2408918284 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40b5880 0x7fbdb40c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.527+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 >> 172.21.15.83:0/2408918284 conn(0x7fbdb40cc690 msgr2=0x7fbdb40cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:35.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.533+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 shutdown_connections 2023-11-28T11:08:35.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.533+0000 7fbdbb563700 1 -- 172.21.15.83:0/2408918284 wait complete. 2023-11-28T11:08:35.536 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T11:08:35.555 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-28T11:08:35.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-28T11:08:35.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.679+0000 7f89c623f700 1 Processor -- start 2023-11-28T11:08:35.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.679+0000 7f89c623f700 1 -- start start 2023-11-28T11:08:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.680+0000 7f89c623f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 0x7f89c00cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.680+0000 7f89c623f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f89c00b1ef0 0x7f89c00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.680+0000 7f89c623f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c015d100 con 0x7f89c006cf30 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.680+0000 7f89c623f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00b2330 con 0x7f89c00b1ef0 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.680+0000 7f89c623f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c006beb0 con 0x7f89c015cd30 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 0x7f89c00cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bffff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f89c00b1ef0 0x7f89c00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 0x7f89c00cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:60614/0 (socket says 172.21.15.83:60614) 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 -- 172.21.15.83:0/4085094399 learned_addr learned my addr 172.21.15.83:0/4085094399 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89beffd700 1 --1- 172.21.15.83:0/4085094399 >> v1:172.21.15.90:6789/0 conn(0x7f89c006cf30 0x7f89c00b17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44386/0 (socket says 172.21.15.83:44386) 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 -- 172.21.15.83:0/4085094399 >> v2:172.21.15.90:3300/0 conn(0x7f89c00b1ef0 msgr2=0x7f89c00cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 --2- 172.21.15.83:0/4085094399 >> v2:172.21.15.90:3300/0 conn(0x7f89c00b1ef0 0x7f89c00cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 -- 172.21.15.83:0/4085094399 >> v1:172.21.15.90:6789/0 conn(0x7f89c006cf30 legacy=0x7f89c00b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 -- 172.21.15.83:0/4085094399 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c006c170 con 0x7f89c015cd30 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89be7fc700 1 -- 172.21.15.83:0/4085094399 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735474665 0 0) 0x7f89a8001110 con 0x7f89c006cf30 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bffff700 1 --2- 172.21.15.83:0/4085094399 >> v2:172.21.15.90:3300/0 conn(0x7f89c00b1ef0 0x7f89c00cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89bf7fe700 1 --2- 172.21.15.83:0/4085094399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 0x7f89c00cae10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89b00098f0 tx=0x7f89b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f357eb7654fefcd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89be7fc700 1 -- 172.21.15.83:0/4085094399 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0011830 con 0x7f89c015cd30 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89be7fc700 1 -- 172.21.15.83:0/4085094399 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0011990 con 0x7f89c015cd30 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.681+0000 7f89be7fc700 1 -- 172.21.15.83:0/4085094399 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0011d20 con 0x7f89c015cd30 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 -- 172.21.15.83:0/4085094399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 msgr2=0x7f89c00cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 --2- 172.21.15.83:0/4085094399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 0x7f89c00cae10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89b00098f0 tx=0x7f89b0009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 -- 172.21.15.83:0/4085094399 shutdown_connections 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 --2- 172.21.15.83:0/4085094399 >> v2:172.21.15.90:3300/0 conn(0x7f89c00b1ef0 0x7f89c00cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 --2- 172.21.15.83:0/4085094399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c015cd30 0x7f89c00cae10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 -- 172.21.15.83:0/4085094399 >> 172.21.15.83:0/4085094399 conn(0x7f89c00cc690 msgr2=0x7f89c00cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 -- 172.21.15.83:0/4085094399 shutdown_connections 2023-11-28T11:08:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.682+0000 7f89c623f700 1 -- 172.21.15.83:0/4085094399 wait complete. 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.683+0000 7f89c623f700 1 Processor -- start 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.683+0000 7f89c623f700 1 -- start start 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89c623f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c006cf30 0x7f89c00adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89c623f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 0x7f89c0149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89c623f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c015cd30 0x7f89c014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89c623f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ad620 con 0x7f89c00b1ef0 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89c623f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c01498f0 con 0x7f89c015cd30 2023-11-28T11:08:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89c623f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00aca10 con 0x7f89c006cf30 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 0x7f89c0149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 0x7f89c0149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48188/0 (socket says 172.21.15.83:48188) 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 -- 172.21.15.83:0/1141703619 learned_addr learned my addr 172.21.15.83:0/1141703619 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89bf7fe700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c006cf30 0x7f89c00adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89bffff700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c015cd30 0x7f89c014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 -- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c006cf30 msgr2=0x7f89c00adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c006cf30 0x7f89c00adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 -- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c015cd30 msgr2=0x7f89c014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c015cd30 0x7f89c014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.684+0000 7f89beffd700 1 -- 172.21.15.83:0/1141703619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b8009a70 con 0x7f89c00b1ef0 2023-11-28T11:08:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.685+0000 7f89beffd700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 0x7f89c0149bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c550 tx=0x7f89a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.685+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8004c80 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.685+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a8004e00 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.685+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b0009550 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.685+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8007310 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.685+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c006c730 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.686+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c006c450 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.686+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f89a801a020 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.686+0000 7f89b7fff700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89a00682f0 0x7f89a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.687+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89a8085650 con 0x7f89c00b1ef0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.687+0000 7f89bf7fe700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89a00682f0 0x7f89a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.687+0000 7f89bf7fe700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89a00682f0 0x7f89a006a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89b0014040 tx=0x7f89b0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.691+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89a8051dd0 con 0x7f89c00b1ef0 2023-11-28T11:08:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:35.884+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f89c006ca10 con 0x7f89c00b1ef0 2023-11-28T11:08:36.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.529+0000 7f89b7fff700 1 -- 172.21.15.83:0/1141703619 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f89a80553f0 con 0x7f89c00b1ef0 2023-11-28T11:08:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89a00682f0 msgr2=0x7f89a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89a00682f0 0x7f89a006a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89b0014040 tx=0x7f89b0008030 comp rx=0 tx=0).stop 2023-11-28T11:08:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 msgr2=0x7f89c0149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 0x7f89c0149bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c550 tx=0x7f89a800c860 comp rx=0 tx=0).stop 2023-11-28T11:08:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 shutdown_connections 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c015cd30 0x7f89c014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.531+0000 7f89c623f700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89c006cf30 0x7f89c00adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.532+0000 7f89c623f700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89a00682f0 0x7f89a006a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.532+0000 7f89c623f700 1 --2- 172.21.15.83:0/1141703619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c00b1ef0 0x7f89c0149bb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.532+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 >> 172.21.15.83:0/1141703619 conn(0x7f89c00cc690 msgr2=0x7f89c00ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.532+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 shutdown_connections 2023-11-28T11:08:36.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.532+0000 7f89c623f700 1 -- 172.21.15.83:0/1141703619 wait complete. 2023-11-28T11:08:36.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:08:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.675+0000 7f6fc6528700 1 Processor -- start 2023-11-28T11:08:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.675+0000 7f6fc6528700 1 -- start start 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.675+0000 7f6fc6528700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 0x7f6fc015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.675+0000 7f6fc6528700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc006bc60 con 0x7f6fc006b480 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.675+0000 7f6fc6528700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc00cadc0 con 0x7f6fc00b16a0 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.675+0000 7f6fc6528700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00cb0a0 con 0x7f6fc015a480 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbf7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6fc006b480 0x7f6fc00c3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46054/0 (socket says 172.21.15.83:46054) 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbf7fe700 1 -- 172.21.15.83:0/1952255350 learned_addr learned my addr 172.21.15.83:0/1952255350 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbffff700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 0x7f6fc015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277795557 0 0) 0x7f6fc006bc60 con 0x7f6fc006b480 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac0035b0 con 0x7f6fc006b480 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394145381 0 0) 0x7f6fc00cadc0 con 0x7f6fc00b16a0 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc006bc60 con 0x7f6fc00b16a0 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1829688075 0 0) 0x7f6fac0035b0 con 0x7f6fc006b480 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 msgr2=0x7f6fc015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 0x7f6fc015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 >> v1:172.21.15.90:6789/0 conn(0x7f6fc00b16a0 legacy=0x7f6fc00cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00cb380 con 0x7f6fc006b480 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6fb0001670 con 0x7f6fc006b480 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 >> v1:172.21.15.83:6789/0 conn(0x7f6fc006b480 legacy=0x7f6fc00c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fac003bc0 0x7f6fac003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fac004850 0x7f6fac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 0x7f6fac005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00cb0a0 con 0x7f6fac003bc0 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac00bba0 con 0x7f6fac004850 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac00bd60 con 0x7f6fc015a480 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbffff700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fac003bc0 0x7f6fac003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fbf7fe700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fac004850 0x7f6fac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.676+0000 7f6fc4ac5700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 0x7f6fac005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbffff700 1 -- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 msgr2=0x7f6fac005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbffff700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fc015a480 0x7f6fac005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbffff700 1 -- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fac004850 msgr2=0x7f6fac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbffff700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fac004850 0x7f6fac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbffff700 1 -- 172.21.15.83:0/1952255350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0035b0 con 0x7f6fac003bc0 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbffff700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fac003bc0 0x7f6fac003fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009b60 tx=0x7f6fac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fed8d692a22da7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800e6c0 con 0x7f6fac003bc0 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa800e820 con 0x7f6fac003bc0 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fbeffd700 1 -- 172.21.15.83:0/1952255350 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800eb90 con 0x7f6fac003bc0 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fc6528700 1 -- 172.21.15.83:0/1952255350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fac003bc0 msgr2=0x7f6fac003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fc6528700 1 --2- 172.21.15.83:0/1952255350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fac003bc0 0x7f6fac003fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009b60 tx=0x7f6fac005880 comp rx=0 tx=0).stop 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.677+0000 7f6fc4ac5700 1 -- 172.21.15.83:0/1952255350 reap_dead start 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.678+0000 7f6fc6528700 1 -- 172.21.15.83:0/1952255350 shutdown_connections 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.678+0000 7f6fc6528700 1 -- 172.21.15.83:0/1952255350 >> 172.21.15.83:0/1952255350 conn(0x7f6fc00cc6a0 msgr2=0x7f6fc00cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.678+0000 7f6fc6528700 1 -- 172.21.15.83:0/1952255350 shutdown_connections 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.678+0000 7f6fc6528700 1 -- 172.21.15.83:0/1952255350 wait complete. 2023-11-28T11:08:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.679+0000 7f6fc6528700 1 Processor -- start 2023-11-28T11:08:36.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.679+0000 7f6fc6528700 1 -- start start 2023-11-28T11:08:36.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fac003bc0 0x7f6fc00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 0x7f6fc00c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc01f2bb0 0x7f6fc01f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00c7f50 con 0x7f6fc01f58e0 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00c8230 con 0x7f6fc01f2bb0 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00c8510 con 0x7f6fac003bc0 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 0x7f6fc00c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 0x7f6fc00c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48208/0 (socket says 172.21.15.83:48208) 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 -- 172.21.15.83:0/4221310220 learned_addr learned my addr 172.21.15.83:0/4221310220 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbffff700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fac003bc0 0x7f6fc00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc4ac5700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc01f2bb0 0x7f6fc01f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 -- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fac003bc0 msgr2=0x7f6fc00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fac003bc0 0x7f6fc00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 -- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc01f2bb0 msgr2=0x7f6fc01f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc01f2bb0 0x7f6fc01f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 -- 172.21.15.83:0/4221310220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc006b8f0 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbf7fe700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 0x7f6fc00c7700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0004f80 tx=0x7f6fb0007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0012560 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb00126e0 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc006bf40 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0012a90 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.680+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc015a850 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.681+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc0074240 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.682+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6fb0012bf0 con 0x7f6fc01f58e0 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.682+0000 7f6fbcff9700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fa00682f0 0x7f6fa006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.682+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f6fb0085d60 con 0x7f6fc01f58e0 2023-11-28T11:08:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.682+0000 7f6fbffff700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fa00682f0 0x7f6fa006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.682+0000 7f6fbffff700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fa00682f0 0x7f6fa006a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80090b0 tx=0x7f6fa8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.686+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6fb00524e0 con 0x7f6fc01f58e0 2023-11-28T11:08:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:36.869+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6fc006d000 con 0x7f6fc01f58e0 2023-11-28T11:08:36.984 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-28T11:08:36.984+0000 7fce36c21700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:08:38.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.544+0000 7f6fbcff9700 1 -- 172.21.15.83:0/4221310220 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f6fb0055b00 con 0x7f6fc01f58e0 2023-11-28T11:08:38.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.549+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fa00682f0 msgr2=0x7f6fa006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.549+0000 7f6fc6528700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fa00682f0 0x7f6fa006a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80090b0 tx=0x7f6fa8008030 comp rx=0 tx=0).stop 2023-11-28T11:08:38.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.549+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 msgr2=0x7f6fc00c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.549+0000 7f6fc6528700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 0x7f6fc00c7700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0004f80 tx=0x7f6fb0007340 comp rx=0 tx=0).stop 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.550+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 shutdown_connections 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.550+0000 7f6fc6528700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc01f2bb0 0x7f6fc01f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.550+0000 7f6fc6528700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fac003bc0 0x7f6fc00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.550+0000 7f6fc6528700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fa00682f0 0x7f6fa006a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.550+0000 7f6fc6528700 1 --2- 172.21.15.83:0/4221310220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc01f58e0 0x7f6fc00c7700 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0004f80 tx=0x7f6fb0007340 comp rx=0 tx=0).stop 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.550+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 >> 172.21.15.83:0/4221310220 conn(0x7f6fc00cc6a0 msgr2=0x7f6fc00c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:38.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.552+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 shutdown_connections 2023-11-28T11:08:38.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.553+0000 7f6fc6528700 1 -- 172.21.15.83:0/4221310220 wait complete. 2023-11-28T11:08:38.555 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data_ec' created 2023-11-28T11:08:38.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-28T11:08:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.697+0000 7f554640f700 1 Processor -- start 2023-11-28T11:08:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.697+0000 7f554640f700 1 -- start start 2023-11-28T11:08:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f554640f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f55401599b0 0x7f55400b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f554640f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55400b5670 0x7f55400b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f554640f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400cb3a0 con 0x7f55400b1c60 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f554640f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554006ccb0 con 0x7f55400b5670 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f554640f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400b5af0 con 0x7f55401599b0 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f55377fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55400b5670 0x7f55400b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f55377fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55400b5670 0x7f55400b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48214/0 (socket says 172.21.15.83:48214) 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.698+0000 7f55377fe700 1 -- 172.21.15.83:0/2091861050 learned_addr learned my addr 172.21.15.83:0/2091861050 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f553ffff700 1 --2- 172.21.15.83:0/2091861050 >> v2:172.21.15.90:3300/0 conn(0x7f55401599b0 0x7f55400b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f55377fe700 1 -- 172.21.15.83:0/2091861050 >> v2:172.21.15.90:3300/0 conn(0x7f55401599b0 msgr2=0x7f55400b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f55377fe700 1 --2- 172.21.15.83:0/2091861050 >> v2:172.21.15.90:3300/0 conn(0x7f55401599b0 0x7f55400b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f55377fe700 1 -- 172.21.15.83:0/2091861050 >> v1:172.21.15.90:6789/0 conn(0x7f55400b1c60 legacy=0x7f55400cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f55377fe700 1 -- 172.21.15.83:0/2091861050 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400cb640 con 0x7f55400b5670 2023-11-28T11:08:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f553ffff700 1 --2- 172.21.15.83:0/2091861050 >> v2:172.21.15.90:3300/0 conn(0x7f55401599b0 0x7f55400b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f55377fe700 1 --2- 172.21.15.83:0/2091861050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55400b5670 0x7f55400b1720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5540071630 tx=0x7f5528009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d279f42f40db94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f553f7fe700 1 -- 172.21.15.83:0/2091861050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800b870 con 0x7f55400b5670 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f553f7fe700 1 -- 172.21.15.83:0/2091861050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552800b9d0 con 0x7f55400b5670 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.699+0000 7f553f7fe700 1 -- 172.21.15.83:0/2091861050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800bd60 con 0x7f55400b5670 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 -- 172.21.15.83:0/2091861050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55400b5670 msgr2=0x7f55400b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 --2- 172.21.15.83:0/2091861050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55400b5670 0x7f55400b1720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5540071630 tx=0x7f5528009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 -- 172.21.15.83:0/2091861050 shutdown_connections 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 --2- 172.21.15.83:0/2091861050 >> v2:172.21.15.90:3300/0 conn(0x7f55401599b0 0x7f55400b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 --2- 172.21.15.83:0/2091861050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55400b5670 0x7f55400b1720 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 -- 172.21.15.83:0/2091861050 >> 172.21.15.83:0/2091861050 conn(0x7f55400cc690 msgr2=0x7f55400cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:38.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 -- 172.21.15.83:0/2091861050 shutdown_connections 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.700+0000 7f554640f700 1 -- 172.21.15.83:0/2091861050 wait complete. 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.701+0000 7f554640f700 1 Processor -- start 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.701+0000 7f554640f700 1 -- start start 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55400b1c60 0x7f5540153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55400b5670 0x7f554015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 0x7f55401f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55401f5fc0 con 0x7f55401599b0 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400b0f70 con 0x7f55400b1c60 2023-11-28T11:08:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554015cb70 con 0x7f55400b5670 2023-11-28T11:08:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 0x7f55401f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f553ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55400b1c60 0x7f5540153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 0x7f55401f63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48224/0 (socket says 172.21.15.83:48224) 2023-11-28T11:08:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 -- 172.21.15.83:0/1968995965 learned_addr learned my addr 172.21.15.83:0/1968995965 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55377fe700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55400b5670 0x7f554015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 -- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55400b5670 msgr2=0x7f554015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55400b5670 0x7f554015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 -- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55400b1c60 msgr2=0x7f5540153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55400b1c60 0x7f5540153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 -- 172.21.15.83:0/1968995965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528009550 con 0x7f55401599b0 2023-11-28T11:08:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f55449ac700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 0x7f55401f63e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f553800cb50 tx=0x7f553800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538003c00 con 0x7f55401599b0 2023-11-28T11:08:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538003d60 con 0x7f55401599b0 2023-11-28T11:08:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553800aa20 con 0x7f55401599b0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.702+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55400cbc00 con 0x7f55401599b0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.703+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55400cb920 con 0x7f55401599b0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.703+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55400cbee0 con 0x7f55401599b0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.704+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5538010050 con 0x7f55401599b0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.704+0000 7f553d7fa700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f55200682f0 0x7f552006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.704+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f5538085e00 con 0x7f55401599b0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.704+0000 7f553ffff700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f55200682f0 0x7f552006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.705+0000 7f553ffff700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f55200682f0 0x7f552006a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55300099d0 tx=0x7f5530008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.709+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5538052390 con 0x7f55401599b0 2023-11-28T11:08:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:38.894+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f55401f6100 con 0x7f55401599b0 2023-11-28T11:08:39.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:39.023 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.543+0000 7f553d7fa700 1 -- 172.21.15.83:0/1968995965 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f55401f6100 con 0x7f55401599b0 2023-11-28T11:08:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.556+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f55200682f0 msgr2=0x7f552006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.556+0000 7f554640f700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f55200682f0 0x7f552006a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55300099d0 tx=0x7f5530008040 comp rx=0 tx=0).stop 2023-11-28T11:08:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.556+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 msgr2=0x7f55401f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.556+0000 7f554640f700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 0x7f55401f63e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f553800cb50 tx=0x7f553800ce60 comp rx=0 tx=0).stop 2023-11-28T11:08:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.557+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 shutdown_connections 2023-11-28T11:08:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.557+0000 7f554640f700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f55200682f0 0x7f552006a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.557+0000 7f554640f700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55401599b0 0x7f55401f63e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.557+0000 7f554640f700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55400b1c60 0x7f5540153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.557+0000 7f554640f700 1 --2- 172.21.15.83:0/1968995965 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55400b5670 0x7f554015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.557+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 >> 172.21.15.83:0/1968995965 conn(0x7f55400cc690 msgr2=0x7f55401557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.559+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 shutdown_connections 2023-11-28T11:08:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.560+0000 7f554640f700 1 -- 172.21.15.83:0/1968995965 wait complete. 2023-11-28T11:08:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 5 allow_ec_overwrites to true 2023-11-28T11:08:39.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.703+0000 7f4c3a76e700 1 Processor -- start 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.703+0000 7f4c3a76e700 1 -- start start 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c3a76e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c3415c890 0x7f4c340aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c3a76e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4c3406e370 0x7f4c3406b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c3a76e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3406d4a0 con 0x7f4c340b4bb0 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c3a76e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340b5020 con 0x7f4c3415c890 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c3a76e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340b5ae0 con 0x7f4c3406e370 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c3415c890 0x7f4c340aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c3415c890 0x7f4c340aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48228/0 (socket says 172.21.15.83:48228) 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 -- 172.21.15.83:0/3111974372 learned_addr learned my addr 172.21.15.83:0/3111974372 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c38d0b700 1 --2- 172.21.15.83:0/3111974372 >> v2:172.21.15.90:3300/0 conn(0x7f4c3406e370 0x7f4c3406b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 -- 172.21.15.83:0/3111974372 >> v2:172.21.15.90:3300/0 conn(0x7f4c3406e370 msgr2=0x7f4c3406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 --2- 172.21.15.83:0/3111974372 >> v2:172.21.15.90:3300/0 conn(0x7f4c3406e370 0x7f4c3406b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 -- 172.21.15.83:0/3111974372 >> v1:172.21.15.90:6789/0 conn(0x7f4c340b4bb0 legacy=0x7f4c3406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 -- 172.21.15.83:0/3111974372 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3406c400 con 0x7f4c3415c890 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c33fff700 1 --2- 172.21.15.83:0/3111974372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340aee60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c34071970 tx=0x7f4c28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1310dd9524d799 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c32ffd700 1 -- 172.21.15.83:0/3111974372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28005e30 con 0x7f4c3415c890 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c32ffd700 1 -- 172.21.15.83:0/3111974372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28004030 con 0x7f4c3415c890 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.704+0000 7f4c32ffd700 1 -- 172.21.15.83:0/3111974372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c280043e0 con 0x7f4c3415c890 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 -- 172.21.15.83:0/3111974372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 msgr2=0x7f4c340aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/3111974372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340aee60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c34071970 tx=0x7f4c28009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 -- 172.21.15.83:0/3111974372 shutdown_connections 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/3111974372 >> v2:172.21.15.90:3300/0 conn(0x7f4c3406e370 0x7f4c3406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/3111974372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340aee60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 -- 172.21.15.83:0/3111974372 >> 172.21.15.83:0/3111974372 conn(0x7f4c340b2b60 msgr2=0x7f4c340b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 -- 172.21.15.83:0/3111974372 shutdown_connections 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.705+0000 7f4c3a76e700 1 -- 172.21.15.83:0/3111974372 wait complete. 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.706+0000 7f4c3a76e700 1 Processor -- start 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.706+0000 7f4c3a76e700 1 -- start start 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c3a76e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c3406e370 0x7f4c341f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c3a76e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4c340b4bb0 0x7f4c34146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c3a76e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c3a76e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3415a810 con 0x7f4c3415c890 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c3a76e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340c9760 con 0x7f4c3406e370 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c3a76e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340c9fc0 con 0x7f4c340b4bb0 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48238/0 (socket says 172.21.15.83:48238) 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 -- 172.21.15.83:0/1992995896 learned_addr learned my addr 172.21.15.83:0/1992995896 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c33fff700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c3406e370 0x7f4c341f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c337fe700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4c340b4bb0 0x7f4c34146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 -- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4c340b4bb0 msgr2=0x7f4c34146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4c340b4bb0 0x7f4c34146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 -- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c3406e370 msgr2=0x7f4c341f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c3406e370 0x7f4c341f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.707+0000 7f4c38d0b700 1 -- 172.21.15.83:0/1992995896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28009500 con 0x7f4c3415c890 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.708+0000 7f4c38d0b700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340c55e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00efc0 tx=0x7f4c2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.708+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c009b20 con 0x7f4c3415c890 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.708+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c00f040 con 0x7f4c3415c890 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.708+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c340b5520 con 0x7f4c3415c890 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.708+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c009e50 con 0x7f4c3415c890 2023-11-28T11:08:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.708+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c340b5240 con 0x7f4c3415c890 2023-11-28T11:08:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.709+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c340b5800 con 0x7f4c3415c890 2023-11-28T11:08:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.709+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c2c014480 con 0x7f4c3415c890 2023-11-28T11:08:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.709+0000 7f4c30ff9700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4c200682f0 0x7f4c2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.710+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f4c2c086cc0 con 0x7f4c3415c890 2023-11-28T11:08:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.710+0000 7f4c33fff700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4c200682f0 0x7f4c2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.710+0000 7f4c33fff700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4c200682f0 0x7f4c2006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009c10 tx=0x7f4c28008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.714+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c2c053100 con 0x7f4c3415c890 2023-11-28T11:08:39.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.943+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f4c3406ac20 con 0x7f4c3415c890 2023-11-28T11:08:39.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.948+0000 7f4c30ff9700 1 -- 172.21.15.83:0/1992995896 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f4c2c0180b0 con 0x7f4c3415c890 2023-11-28T11:08:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4c200682f0 msgr2=0x7f4c2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4c200682f0 0x7f4c2006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009c10 tx=0x7f4c28008070 comp rx=0 tx=0).stop 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 msgr2=0x7f4c340c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340c55e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00efc0 tx=0x7f4c2c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 shutdown_connections 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4c200682f0 0x7f4c2006a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3415c890 0x7f4c340c55e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c3406e370 0x7f4c341f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 --2- 172.21.15.83:0/1992995896 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4c340b4bb0 0x7f4c34146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 >> 172.21.15.83:0/1992995896 conn(0x7f4c340b2b60 msgr2=0x7f4c340ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 shutdown_connections 2023-11-28T11:08:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:39.951+0000 7f4c3a76e700 1 -- 172.21.15.83:0/1992995896 wait complete. 2023-11-28T11:08:39.954 INFO:teuthology.orchestra.run.smithi083.stderr:added data pool 5 to fsmap 2023-11-28T11:08:39.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:40.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.084+0000 7fc6b590f700 1 Processor -- start 2023-11-28T11:08:40.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.084+0000 7fc6b590f700 1 -- start start 2023-11-28T11:08:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.085+0000 7fc6b590f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc6b015a510 0x7fc6b00cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6b590f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6b015a8e0 0x7fc6b00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6b590f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b015f1d0 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6b590f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00cd780 con 0x7fc6b015a8e0 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6b590f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b015eef0 con 0x7fc6b015a510 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6af7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6b015a8e0 0x7fc6b00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6ae7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc6b00cfdf0 0x7fc6b00b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46102/0 (socket says 172.21.15.83:46102) 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6ae7fc700 1 -- 172.21.15.83:0/1056203352 learned_addr learned my addr 172.21.15.83:0/1056203352 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6aeffd700 1 --2- 172.21.15.83:0/1056203352 >> v2:172.21.15.90:3300/0 conn(0x7fc6b015a510 0x7fc6b00cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6af7fe700 1 -- 172.21.15.83:0/1056203352 >> v2:172.21.15.90:3300/0 conn(0x7fc6b015a510 msgr2=0x7fc6b00cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6af7fe700 1 --2- 172.21.15.83:0/1056203352 >> v2:172.21.15.90:3300/0 conn(0x7fc6b015a510 0x7fc6b00cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6af7fe700 1 -- 172.21.15.83:0/1056203352 >> v1:172.21.15.83:6789/0 conn(0x7fc6b00cfdf0 legacy=0x7fc6b00b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6adffb700 1 -- 172.21.15.83:0/1056203352 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820091545 0 0) 0x7fc6b015f1d0 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6af7fe700 1 -- 172.21.15.83:0/1056203352 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00cedd0 con 0x7fc6b015a8e0 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6aeffd700 1 --2- 172.21.15.83:0/1056203352 >> v2:172.21.15.90:3300/0 conn(0x7fc6b015a510 0x7fc6b00cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6af7fe700 1 --2- 172.21.15.83:0/1056203352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b015a8e0 0x7fc6b00b3d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00ce6b0 tx=0x7fc6a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b0900754bfdcbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6adffb700 1 -- 172.21.15.83:0/1056203352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800e070 con 0x7fc6b015a8e0 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6adffb700 1 -- 172.21.15.83:0/1056203352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a800bbd0 con 0x7fc6b015a8e0 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.086+0000 7fc6adffb700 1 -- 172.21.15.83:0/1056203352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8012430 con 0x7fc6b015a8e0 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 -- 172.21.15.83:0/1056203352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b015a8e0 msgr2=0x7fc6b00b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1056203352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b015a8e0 0x7fc6b00b3d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00ce6b0 tx=0x7fc6a800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 -- 172.21.15.83:0/1056203352 shutdown_connections 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1056203352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b015a8e0 0x7fc6b00b3d30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1056203352 >> v2:172.21.15.90:3300/0 conn(0x7fc6b015a510 0x7fc6b00cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 -- 172.21.15.83:0/1056203352 >> 172.21.15.83:0/1056203352 conn(0x7fc6b00cd960 msgr2=0x7fc6b00cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 -- 172.21.15.83:0/1056203352 shutdown_connections 2023-11-28T11:08:40.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.087+0000 7fc6b590f700 1 -- 172.21.15.83:0/1056203352 wait complete. 2023-11-28T11:08:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.088+0000 7fc6b590f700 1 Processor -- start 2023-11-28T11:08:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.088+0000 7fc6b590f700 1 -- start start 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6b590f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 0x7fc6b00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6b590f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6b015a510 0x7fc6b00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6b590f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6b015a8e0 0x7fc6b01571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6b590f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00b5540 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6b590f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00c9700 con 0x7fc6b015a510 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6b590f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00b5770 con 0x7fc6b015a8e0 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 0x7fc6b00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 0x7fc6b00c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48246/0 (socket says 172.21.15.83:48246) 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 -- 172.21.15.83:0/1722704590 learned_addr learned my addr 172.21.15.83:0/1722704590 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6ae7fc700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6b015a510 0x7fc6b00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6af7fe700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6b015a8e0 0x7fc6b01571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 -- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6b015a8e0 msgr2=0x7fc6b01571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6b015a8e0 0x7fc6b01571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 -- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6b015a510 msgr2=0x7fc6b00b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6b015a510 0x7fc6b00b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 -- 172.21.15.83:0/1722704590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009a70 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6af7fe700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6b015a8e0 0x7fc6b01571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc6aeffd700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 0x7fc6b00c7580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c750 tx=0x7fc6a400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.089+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4011800 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.090+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4011960 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.090+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4011cd0 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.090+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b00cf390 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.090+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b00cf0b0 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.090+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b00b3070 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.091+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6a4020070 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.091+0000 7fc697fff700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc69c0682f0 0x7fc69c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.091+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc6a4085d60 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.091+0000 7fc6ae7fc700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc69c0682f0 0x7fc69c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.092+0000 7fc6ae7fc700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc69c0682f0 0x7fc69c06a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6b01f34c0 tx=0x7fc6a0007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.095+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6a40522d0 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.320+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6b0074320 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.321+0000 7fc697fff700 1 -- 172.21.15.83:0/1722704590 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fc6a40558f0 con 0x7fc6b00cfdf0 2023-11-28T11:08:40.322 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:40.322 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:39.945283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T11:08:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc69c0682f0 msgr2=0x7fc69c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc69c0682f0 0x7fc69c06a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6b01f34c0 tx=0x7fc6a0007e70 comp rx=0 tx=0).stop 2023-11-28T11:08:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 msgr2=0x7fc6b00c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 0x7fc6b00c7580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c750 tx=0x7fc6a400ca60 comp rx=0 tx=0).stop 2023-11-28T11:08:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 shutdown_connections 2023-11-28T11:08:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6b015a8e0 0x7fc6b01571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc69c0682f0 0x7fc69c06a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b00cfdf0 0x7fc6b00c7580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 --2- 172.21.15.83:0/1722704590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6b015a510 0x7fc6b00b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 >> 172.21.15.83:0/1722704590 conn(0x7fc6b00cd960 msgr2=0x7fc6b00b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 shutdown_connections 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.323+0000 7fc6b590f700 1 -- 172.21.15.83:0/1722704590 wait complete. 2023-11-28T11:08:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2023-11-28T11:08:40.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.454+0000 7fd7f7c10700 1 Processor -- start 2023-11-28T11:08:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.454+0000 7fd7f7c10700 1 -- start start 2023-11-28T11:08:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f7c10700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f015a510 0x7fd7f00cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f7c10700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd7f00cfdf0 0x7fd7f00b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f7c10700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f015f1d0 con 0x7fd7f015a8e0 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f7c10700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00cd780 con 0x7fd7f015a510 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f7c10700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f015eef0 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f015a510 0x7fd7f00cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f015a510 0x7fd7f00cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48254/0 (socket says 172.21.15.83:48254) 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3419799804 learned_addr learned my addr 172.21.15.83:0/3419799804 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f61ad700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd7f015a8e0 0x7fd7f00b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46118/0 (socket says 172.21.15.83:46118) 2023-11-28T11:08:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f51ab700 1 --2- 172.21.15.83:0/3419799804 >> v2:172.21.15.90:3300/0 conn(0x7fd7f00cfdf0 0x7fd7f00b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3419799804 >> v2:172.21.15.90:3300/0 conn(0x7fd7f00cfdf0 msgr2=0x7fd7f00b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 --2- 172.21.15.83:0/3419799804 >> v2:172.21.15.90:3300/0 conn(0x7fd7f00cfdf0 0x7fd7f00b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3419799804 >> v1:172.21.15.83:6789/0 conn(0x7fd7f015a8e0 legacy=0x7fd7f00b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3419799804 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00cedd0 con 0x7fd7f015a510 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f49aa700 1 -- 172.21.15.83:0/3419799804 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113512168 0 0) 0x7fd7f015f1d0 con 0x7fd7f015a8e0 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.455+0000 7fd7f59ac700 1 --2- 172.21.15.83:0/3419799804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f015a510 0x7fd7f00cf8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7f004e880 tx=0x7fd7ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf520a1068e0bf6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f49aa700 1 -- 172.21.15.83:0/3419799804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00b870 con 0x7fd7f015a510 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f49aa700 1 -- 172.21.15.83:0/3419799804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec00b9d0 con 0x7fd7f015a510 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f49aa700 1 -- 172.21.15.83:0/3419799804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00bd60 con 0x7fd7f015a510 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3419799804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f015a510 msgr2=0x7fd7f00cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3419799804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f015a510 0x7fd7f00cf8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7f004e880 tx=0x7fd7ec009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3419799804 shutdown_connections 2023-11-28T11:08:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3419799804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f015a510 0x7fd7f00cf8b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3419799804 >> v2:172.21.15.90:3300/0 conn(0x7fd7f00cfdf0 0x7fd7f00b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3419799804 >> 172.21.15.83:0/3419799804 conn(0x7fd7f00cd960 msgr2=0x7fd7f00cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3419799804 shutdown_connections 2023-11-28T11:08:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.456+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3419799804 wait complete. 2023-11-28T11:08:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.458+0000 7fd7f7c10700 1 Processor -- start 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.458+0000 7fd7f7c10700 1 -- start start 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 0x7fd7f006f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f015a510 0x7fd7f006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f015a8e0 0x7fd7f006d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f006dc90 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01f2f60 con 0x7fd7f015a510 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f015d150 con 0x7fd7f015a8e0 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 0x7fd7f006f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 0x7fd7f006f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48260/0 (socket says 172.21.15.83:48260) 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3696914996 learned_addr learned my addr 172.21.15.83:0/3696914996 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3696914996 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f015a8e0 msgr2=0x7fd7f006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f015a8e0 0x7fd7f006d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3696914996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f015a510 msgr2=0x7fd7f006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f015a510 0x7fd7f006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 -- 172.21.15.83:0/3696914996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec009550 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f59ac700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 0x7fd7f006f3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009c10 tx=0x7fd7ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec004310 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec01f070 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00cf390 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0036a0 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.459+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00cf0b0 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.460+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f00b4440 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.461+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7ec00f050 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.461+0000 7fd7e67fc700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7d40682f0 0x7fd7d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.461+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd7ec0874a0 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.461+0000 7fd7f51ab700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7d40682f0 0x7fd7d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.461+0000 7fd7f51ab700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7d40682f0 0x7fd7d406a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00c7560 tx=0x7fd7e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.465+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7ec053990 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.650+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f0074320 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.651+0000 7fd7e67fc700 1 -- 172.21.15.83:0/3696914996 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fd7ec056fb0 con 0x7fd7f00cfdf0 2023-11-28T11:08:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:40.653 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7d40682f0 msgr2=0x7fd7d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7d40682f0 0x7fd7d406a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00c7560 tx=0x7fd7e0008040 comp rx=0 tx=0).stop 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 msgr2=0x7fd7f006f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 0x7fd7f006f3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009c10 tx=0x7fd7ec004750 comp rx=0 tx=0).stop 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 shutdown_connections 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f015a8e0 0x7fd7f006d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7d40682f0 0x7fd7d406a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f00cfdf0 0x7fd7f006f3d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 --2- 172.21.15.83:0/3696914996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f015a510 0x7fd7f006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 >> 172.21.15.83:0/3696914996 conn(0x7fd7f00cd960 msgr2=0x7fd7f00cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 shutdown_connections 2023-11-28T11:08:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.654+0000 7fd7f7c10700 1 -- 172.21.15.83:0/3696914996 wait complete. 2023-11-28T11:08:40.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:40.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.786+0000 7f9023246700 1 Processor -- start 2023-11-28T11:08:40.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.786+0000 7f9023246700 1 -- start start 2023-11-28T11:08:40.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9023246700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f901c15cd10 0x7f901c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9023246700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f901c15d0e0 0x7f901c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9023246700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c0b18a0 0x7f901c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9023246700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0b2280 con 0x7f901c15cd10 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9023246700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0cbfb0 con 0x7f901c15d0e0 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9023246700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0cb4e0 con 0x7f901c0b18a0 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f901c15cd10 0x7f901c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f901c15cd10 0x7f901c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48268/0 (socket says 172.21.15.83:48268) 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 -- 172.21.15.83:0/1221535827 learned_addr learned my addr 172.21.15.83:0/1221535827 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 -- 172.21.15.83:0/1221535827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c0b18a0 msgr2=0x7f901c0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f901bfff700 1 --2- 172.21.15.83:0/1221535827 >> v2:172.21.15.90:3300/0 conn(0x7f901c15d0e0 0x7f901c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f90217e3700 1 --2- 172.21.15.83:0/1221535827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c0b18a0 0x7f901c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 --2- 172.21.15.83:0/1221535827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c0b18a0 0x7f901c0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 -- 172.21.15.83:0/1221535827 >> v2:172.21.15.90:3300/0 conn(0x7f901c15d0e0 msgr2=0x7f901c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 --2- 172.21.15.83:0/1221535827 >> v2:172.21.15.90:3300/0 conn(0x7f901c15d0e0 0x7f901c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 -- 172.21.15.83:0/1221535827 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0cb710 con 0x7f901c15cd10 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f9020fe2700 1 --2- 172.21.15.83:0/1221535827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c15cd10 0x7f901c0ca970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f901c0cc800 tx=0x7f9010009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec4bf6fd32981 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.787+0000 7f901b7fe700 1 -- 172.21.15.83:0/1221535827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000b870 con 0x7f901c15cd10 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f901b7fe700 1 -- 172.21.15.83:0/1221535827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901000b9d0 con 0x7f901c15cd10 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f901b7fe700 1 -- 172.21.15.83:0/1221535827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000bd60 con 0x7f901c15cd10 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 -- 172.21.15.83:0/1221535827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c15cd10 msgr2=0x7f901c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 --2- 172.21.15.83:0/1221535827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c15cd10 0x7f901c0ca970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f901c0cc800 tx=0x7f9010009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 -- 172.21.15.83:0/1221535827 shutdown_connections 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 --2- 172.21.15.83:0/1221535827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c0b18a0 0x7f901c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 --2- 172.21.15.83:0/1221535827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c15cd10 0x7f901c0ca970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 --2- 172.21.15.83:0/1221535827 >> v2:172.21.15.90:3300/0 conn(0x7f901c15d0e0 0x7f901c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 -- 172.21.15.83:0/1221535827 >> 172.21.15.83:0/1221535827 conn(0x7f901c06e9a0 msgr2=0x7f901c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 -- 172.21.15.83:0/1221535827 shutdown_connections 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.788+0000 7f9023246700 1 -- 172.21.15.83:0/1221535827 wait complete. 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.789+0000 7f9023246700 1 Processor -- start 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.789+0000 7f9023246700 1 -- start start 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9023246700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 0x7f901c1f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9023246700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f901c15cd10 0x7f901c15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9023246700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c15d0e0 0x7f901c0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9023246700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0c7a30 con 0x7f901c0b18a0 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9023246700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c155c60 con 0x7f901c15cd10 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9023246700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c160700 con 0x7f901c15d0e0 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9020fe2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 0x7f901c1f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9020fe2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 0x7f901c1f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48274/0 (socket says 172.21.15.83:48274) 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f9020fe2700 1 -- 172.21.15.83:0/1715907167 learned_addr learned my addr 172.21.15.83:0/1715907167 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f901bfff700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f901c15cd10 0x7f901c15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.790+0000 7f90217e3700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c15d0e0 0x7f901c0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9020fe2700 1 -- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c15d0e0 msgr2=0x7f901c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9020fe2700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c15d0e0 0x7f901c0c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9020fe2700 1 -- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f901c15cd10 msgr2=0x7f901c15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9020fe2700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f901c15cd10 0x7f901c15fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9020fe2700 1 -- 172.21.15.83:0/1715907167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9010009550 con 0x7f901c0b18a0 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f901bfff700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f901c15cd10 0x7f901c15fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9020fe2700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 0x7f901c1f3a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f901c06dc40 tx=0x7f9010004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010004310 con 0x7f901c0b18a0 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901001f070 con 0x7f901c0b18a0 2023-11-28T11:08:40.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90100036a0 con 0x7f901c0b18a0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c0cbcd0 con 0x7f901c0b18a0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.791+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c0cb9f0 con 0x7f901c0b18a0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.792+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c06b9b0 con 0x7f901c0b18a0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.792+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f901002a020 con 0x7f901c0b18a0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.792+0000 7f90197fa700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90000682f0 0x7f900006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.793+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f90100876c0 con 0x7f901c0b18a0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.793+0000 7f901bfff700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90000682f0 0x7f900006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.793+0000 7f901bfff700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90000682f0 0x7f900006a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f901c0c9bd0 tx=0x7f9008008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.797+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9010053b40 con 0x7f901c0b18a0 2023-11-28T11:08:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.985+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f901c0c7ca0 con 0x7f901c0b18a0 2023-11-28T11:08:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.987+0000 7f90197fa700 1 -- 172.21.15.83:0/1715907167 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f9010057160 con 0x7f901c0b18a0 2023-11-28T11:08:40.988 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:40.989 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:40.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90000682f0 msgr2=0x7f900006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90000682f0 0x7f900006a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f901c0c9bd0 tx=0x7f9008008040 comp rx=0 tx=0).stop 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 msgr2=0x7f901c1f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 0x7f901c1f3a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f901c06dc40 tx=0x7f9010004750 comp rx=0 tx=0).stop 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 shutdown_connections 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f901c15d0e0 0x7f901c0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90000682f0 0x7f900006a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901c0b18a0 0x7f901c1f3a50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 --2- 172.21.15.83:0/1715907167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f901c15cd10 0x7f901c15fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 >> 172.21.15.83:0/1715907167 conn(0x7f901c06e9a0 msgr2=0x7f901c0ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 shutdown_connections 2023-11-28T11:08:40.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:40.989+0000 7f9023246700 1 -- 172.21.15.83:0/1715907167 wait complete. 2023-11-28T11:08:41.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:41.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.135+0000 7fd73029d700 1 Processor -- start 2023-11-28T11:08:41.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.135+0000 7fd73029d700 1 -- start start 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd73029d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd72815cd10 0x7fd7280ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd73029d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd72815d0e0 0x7fd7280cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd73029d700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280cbfb0 con 0x7fd7280b18a0 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd73029d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72806cee0 con 0x7fd72815cd10 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd73029d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280cb710 con 0x7fd72815d0e0 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd72815cd10 0x7fd7280ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd72815cd10 0x7fd7280ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48282/0 (socket says 172.21.15.83:48282) 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 -- 172.21.15.83:0/4274406537 learned_addr learned my addr 172.21.15.83:0/4274406537 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72d838700 1 --2- 172.21.15.83:0/4274406537 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd72815d0e0 0x7fd7280cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 -- 172.21.15.83:0/4274406537 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd72815d0e0 msgr2=0x7fd7280cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 --2- 172.21.15.83:0/4274406537 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd72815d0e0 0x7fd7280cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 -- 172.21.15.83:0/4274406537 >> v1:172.21.15.90:6789/0 conn(0x7fd7280b18a0 legacy=0x7fd7281609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 -- 172.21.15.83:0/4274406537 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280cb4e0 con 0x7fd72815cd10 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72e039700 1 --2- 172.21.15.83:0/4274406537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280ca970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098f0 tx=0x7fd724009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd002065b2c4456 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72d037700 1 -- 172.21.15.83:0/4274406537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72400b8a0 con 0x7fd72815cd10 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72d037700 1 -- 172.21.15.83:0/4274406537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72400ba00 con 0x7fd72815cd10 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.136+0000 7fd72d037700 1 -- 172.21.15.83:0/4274406537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72400bd90 con 0x7fd72815cd10 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 -- 172.21.15.83:0/4274406537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 msgr2=0x7fd7280ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 --2- 172.21.15.83:0/4274406537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280ca970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098f0 tx=0x7fd724009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 -- 172.21.15.83:0/4274406537 shutdown_connections 2023-11-28T11:08:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 --2- 172.21.15.83:0/4274406537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280ca970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 --2- 172.21.15.83:0/4274406537 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd72815d0e0 0x7fd7280cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 -- 172.21.15.83:0/4274406537 >> 172.21.15.83:0/4274406537 conn(0x7fd72806e9a0 msgr2=0x7fd7280cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 -- 172.21.15.83:0/4274406537 shutdown_connections 2023-11-28T11:08:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.137+0000 7fd73029d700 1 -- 172.21.15.83:0/4274406537 wait complete. 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.138+0000 7fd73029d700 1 Processor -- start 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.138+0000 7fd73029d700 1 -- start start 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd73029d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7280b18a0 0x7fd7280c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd73029d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd73029d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72815d0e0 0x7fd7280c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd73029d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72814de10 con 0x7fd72815cd10 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd73029d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7281f2630 con 0x7fd72815d0e0 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd73029d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280b53b0 con 0x7fd7280b18a0 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48290/0 (socket says 172.21.15.83:48290) 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 -- 172.21.15.83:0/581501367 learned_addr learned my addr 172.21.15.83:0/581501367 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72e039700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7280b18a0 0x7fd7280c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72e83a700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72815d0e0 0x7fd7280c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 -- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7280b18a0 msgr2=0x7fd7280c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7280b18a0 0x7fd7280c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 -- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72815d0e0 msgr2=0x7fd7280c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72815d0e0 0x7fd7280c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.139+0000 7fd72d838700 1 -- 172.21.15.83:0/581501367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724009550 con 0x7fd72815cd10 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.140+0000 7fd72d838700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280b4b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd71800d800 tx=0x7fd71800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.140+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180126b0 con 0x7fd72815cd10 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.140+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718012810 con 0x7fd72815cd10 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.140+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718012bc0 con 0x7fd72815cd10 2023-11-28T11:08:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.140+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7280cbcd0 con 0x7fd72815cd10 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.140+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7280cb9f0 con 0x7fd72815cd10 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.141+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72806b9b0 con 0x7fd72815cd10 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.141+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd718012d20 con 0x7fd72815cd10 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.141+0000 7fd71effd700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd70c0682f0 0x7fd70c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.142+0000 7fd72e039700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd70c0682f0 0x7fd70c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.142+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd718086240 con 0x7fd72815cd10 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.142+0000 7fd72e039700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd70c0682f0 0x7fd70c06a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd724009c40 tx=0x7fd724008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.146+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7180527b0 con 0x7fd72815cd10 2023-11-28T11:08:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.368+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd728074320 con 0x7fd72815cd10 2023-11-28T11:08:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.368+0000 7fd71effd700 1 -- 172.21.15.83:0/581501367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fd718055dd0 con 0x7fd72815cd10 2023-11-28T11:08:41.370 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:41.371 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:39.945283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T11:08:41.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd70c0682f0 msgr2=0x7fd70c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd70c0682f0 0x7fd70c06a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd724009c40 tx=0x7fd724008030 comp rx=0 tx=0).stop 2023-11-28T11:08:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 msgr2=0x7fd7280b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280b4b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd71800d800 tx=0x7fd71800db10 comp rx=0 tx=0).stop 2023-11-28T11:08:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 shutdown_connections 2023-11-28T11:08:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72815d0e0 0x7fd7280c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7280b18a0 0x7fd7280c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd70c0682f0 0x7fd70c06a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 --2- 172.21.15.83:0/581501367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd72815cd10 0x7fd7280b4b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 >> 172.21.15.83:0/581501367 conn(0x7fd72806e9a0 msgr2=0x7fd72815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 shutdown_connections 2023-11-28T11:08:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.370+0000 7fd73029d700 1 -- 172.21.15.83:0/581501367 wait complete. 2023-11-28T11:08:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2023-11-28T11:08:41.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:08:35.518363+0000', 'modified': '2023-11-28T11:08:39.945283+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/3706545143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 3706545143}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 3706545143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-28T11:08:41.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:08:41.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T11:08:41.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.909+0000 7f9a1d5d3700 1 Processor -- start 2023-11-28T11:08:41.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.909+0000 7f9a1d5d3700 1 -- start start 2023-11-28T11:08:41.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.909+0000 7f9a1d5d3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a180d0e40 0x7f9a180d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.909+0000 7f9a1d5d3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a180b61c0 con 0x7f9a180b5d50 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.909+0000 7f9a1d5d3700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a180d27c0 con 0x7f9a180b4980 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.909+0000 7f9a1d5d3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180cd8c0 con 0x7f9a180d0e40 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a180d0e40 0x7f9a180d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a180d0e40 0x7f9a180d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48298/0 (socket says 172.21.15.83:48298) 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 -- 172.21.15.83:0/1005506713 learned_addr learned my addr 172.21.15.83:0/1005506713 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 -- 172.21.15.83:0/1005506713 >> v1:172.21.15.90:6789/0 conn(0x7f9a180b4980 legacy=0x7f9a18160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 -- 172.21.15.83:0/1005506713 >> v1:172.21.15.83:6789/0 conn(0x7f9a180b5d50 legacy=0x7f9a180b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 -- 172.21.15.83:0/1005506713 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180d2a60 con 0x7f9a180d0e40 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a15ffb700 1 -- 172.21.15.83:0/1005506713 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449651717 0 0) 0x7f9a180b61c0 con 0x7f9a180b5d50 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a177fe700 1 --2- 172.21.15.83:0/1005506713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180d0e40 0x7f9a180d1280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009e50 tx=0x7f9a1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398beb783bf178a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a15ffb700 1 -- 172.21.15.83:0/1005506713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000e070 con 0x7f9a180d0e40 2023-11-28T11:08:41.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a15ffb700 1 -- 172.21.15.83:0/1005506713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1000b9c0 con 0x7f9a180d0e40 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.910+0000 7f9a15ffb700 1 -- 172.21.15.83:0/1005506713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000bd70 con 0x7f9a180d0e40 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/1005506713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180d0e40 msgr2=0x7f9a180d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/1005506713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180d0e40 0x7f9a180d1280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009e50 tx=0x7f9a1000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/1005506713 shutdown_connections 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/1005506713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180d0e40 0x7f9a180d1280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/1005506713 >> 172.21.15.83:0/1005506713 conn(0x7f9a1806ceb0 msgr2=0x7f9a180cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/1005506713 shutdown_connections 2023-11-28T11:08:41.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.911+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/1005506713 wait complete. 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.912+0000 7f9a1d5d3700 1 Processor -- start 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.912+0000 7f9a1d5d3700 1 -- start start 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a1d5d3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 0x7f9a18155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a1d5d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a180b5d50 0x7f9a18152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a1d5d3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a180d0e40 0x7f9a180ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a1d5d3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18153040 con 0x7f9a180b4980 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a1d5d3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18156f20 con 0x7f9a180b5d50 2023-11-28T11:08:41.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a1d5d3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18157200 con 0x7f9a180d0e40 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 0x7f9a18155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 0x7f9a18155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48302/0 (socket says 172.21.15.83:48302) 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2196370299 learned_addr learned my addr 172.21.15.83:0/2196370299 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a180b5d50 0x7f9a18152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a177fe700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a180d0e40 0x7f9a180ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a180d0e40 msgr2=0x7f9a180ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a180d0e40 0x7f9a180ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a180b5d50 msgr2=0x7f9a18152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a180b5d50 0x7f9a18152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2196370299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10009b30 con 0x7f9a180b4980 2023-11-28T11:08:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a177fe700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a180d0e40 0x7f9a180ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a180b5d50 0x7f9a18152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.913+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 0x7f9a18155e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c450 tx=0x7f9a0800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.914+0000 7f99ff7fe700 1 -- 172.21.15.83:0/2196370299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080052d0 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.914+0000 7f99ff7fe700 1 -- 172.21.15.83:0/2196370299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08005430 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.914+0000 7f99ff7fe700 1 -- 172.21.15.83:0/2196370299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080038c0 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.914+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a180cd550 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.914+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a180cdba0 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.915+0000 7f99ff7fe700 1 -- 172.21.15.83:0/2196370299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a08019030 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.916+0000 7f99ff7fe700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a040682f0 0x7f9a0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.916+0000 7f99ff7fe700 1 -- 172.21.15.83:0/2196370299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a080852c0 con 0x7f9a180b4980 2023-11-28T11:08:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.916+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a040682f0 0x7f9a0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.916+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a040682f0 0x7f9a0406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a180cd520 tx=0x7f9a0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 Processor -- start 2023-11-28T11:08:41.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 -- start start 2023-11-28T11:08:41.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109ca93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109caaf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109c953bc0 con 0x56109ca93810 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109c9c2350 con 0x56109ca91130 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.918+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56109c9bb880 con 0x56109ca94120 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca69a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48308/0 (socket says 172.21.15.83:48308) 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2256702516 learned_addr learned my addr 172.21.15.83:0/2256702516 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109ca93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 msgr2=0x56109caaf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a177fe700 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109caaf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109caaf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 msgr2=0x56109ca93be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109ca93be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 -- 172.21.15.83:0/2256702516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56109c83d5e0 con 0x56109ca93810 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109ca93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca69a20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a0805d050 tx=0x7f9a08054e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48e5bc20a1d9279 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f99fcff9700 1 -- 172.21.15.83:0/2256702516 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080591b0 con 0x56109ca93810 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f99fcff9700 1 -- 172.21.15.83:0/2256702516 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08053800 con 0x56109ca93810 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.919+0000 7f99fcff9700 1 -- 172.21.15.83:0/2256702516 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0805d2f0 con 0x56109ca93810 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2256702516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 msgr2=0x56109ca69a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca69a20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a0805d050 tx=0x7f9a08054e40 comp rx=0 tx=0).stop 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2256702516 shutdown_connections 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109caaf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca69a20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2256702516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109ca93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2256702516 >> 172.21.15.83:0/2256702516 conn(0x56109ca66d60 msgr2=0x56109ca676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2256702516 shutdown_connections 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2256702516 wait complete. 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.920+0000 7f9a2975cb80 1 -- start start 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 Processor -- start 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca92b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109cab4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109c83f8d0 con 0x56109ca93810 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109cab5370 con 0x56109ca94120 2023-11-28T11:08:41.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56109ca8c2d0 con 0x56109ca91130 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a2975cb80 10 client.0 _ll_register_callbacks cb 0x56109c861240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca92b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca92b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48316/0 (socket says 172.21.15.83:48316) 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.921+0000 7f9a167fc700 1 -- 172.21.15.83:0/2505877619 learned_addr learned my addr 172.21.15.83:0/2505877619 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a177fe700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109cab4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a167fc700 1 -- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 msgr2=0x56109ca925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a167fc700 1 -- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 msgr2=0x56109cab4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109cab4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a167fc700 1 -- 172.21.15.83:0/2505877619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a180cdba0 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a167fc700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca92b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c003900 tx=0x7f9a0c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1468b7577c4858f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f99f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f99f5ffb700 1 -- 172.21.15.83:0/2505877619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c015110 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f99f5ffb700 1 -- 172.21.15.83:0/2505877619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a0c015270 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56109c989140 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f99f5ffb700 1 -- 172.21.15.83:0/2505877619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a0c0158d0 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 10 client.4419 fetch_fsmap learned FSMap version 7 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56109c7e9d90 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f99f5ffb700 1 -- 172.21.15.83:0/2505877619 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56109c989140 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f99f5ffb700 1 -- 172.21.15.83:0/2505877619 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9a0c015a30 con 0x56109ca93810 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 10 client.4419 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 10 client.4419 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:08:41.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 20 client.4419 populate_metadata read hostname 'smithi083' 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109cac9620 0x56109cacba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 4 client.4419 mds_command: new command op to 4263 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 --> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56109c83ff10 con 0x56109cac9620 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.922+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109cac9620 0x56109cacba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f99f5ffb700 10 client.4419 ms_handle_connect on v2:172.21.15.83:6835/3706545143 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109cac9620 0x56109cacba00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54caefd65b9b90f server_cookie=5d289e7ab6d826b6 in_seq=0 out_seq=0 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f99f5ffb700 1 -- 172.21.15.83:0/2505877619 <== mds.0 v2:172.21.15.83:6835/3706545143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56109c83ff10 con 0x56109cac9620 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f99f5ffb700 10 client.4419 handle_command_reply: tid=0 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 client.4419 shutdown 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 msgr2=0x56109ca92b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca92b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c003900 tx=0x7f9a0c007bc0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 shutdown_connections 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109cac9620 0x56109cacba00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109cab4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/2505877619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca92b30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f99f5ffb700 0 client.4419 ms_handle_reset on v2:172.21.15.83:6835/3706545143 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.923+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 >> 172.21.15.83:0/2505877619 conn(0x56109ca66d60 msgr2=0x56109ca69a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.924+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 shutdown_connections 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.924+0000 7f9a2975cb80 1 -- 172.21.15.83:0/2505877619 wait complete. 2023-11-28T11:08:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.924+0000 7f9a2975cb80 20 client.4419 trim_cache size 0 max 16384 2023-11-28T11:08:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.941+0000 7f9a2975cb80 1 Processor -- start 2023-11-28T11:08:41.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.941+0000 7f9a2975cb80 1 -- start start 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca68230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109ca68770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ca664e0 con 0x56109ca93810 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ca69f60 con 0x56109ca94120 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56109ca60060 con 0x56109ca91130 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca68230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca68230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48324/0 (socket says 172.21.15.83:48324) 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 -- 172.21.15.83:0/803500711 learned_addr learned my addr 172.21.15.83:0/803500711 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a177fe700 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109ca68770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 -- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 msgr2=0x56109ca67130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca67130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 -- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 msgr2=0x56109ca68770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109ca68770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 -- 172.21.15.83:0/803500711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56109c982140 con 0x56109ca93810 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a177fe700 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109ca68770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f9a167fc700 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca68230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c015f20 tx=0x7f9a0c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5910a378c2778def server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f99fd7fa700 1 -- 172.21.15.83:0/803500711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0151d0 con 0x56109ca93810 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f99fd7fa700 1 -- 172.21.15.83:0/803500711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c01b050 con 0x56109ca93810 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.942+0000 7f99fd7fa700 1 -- 172.21.15.83:0/803500711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c015920 con 0x56109ca93810 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 -- 172.21.15.83:0/803500711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 msgr2=0x56109ca68230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca68230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c015f20 tx=0x7f9a0c01e000 comp rx=0 tx=0).stop 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 -- 172.21.15.83:0/803500711 shutdown_connections 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca94120 0x56109ca68770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca91130 0x56109ca67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/803500711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca68230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 -- 172.21.15.83:0/803500711 >> 172.21.15.83:0/803500711 conn(0x56109ca66d60 msgr2=0x56109ca6a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 -- 172.21.15.83:0/803500711 shutdown_connections 2023-11-28T11:08:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 -- 172.21.15.83:0/803500711 wait complete. 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 -- start start 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.943+0000 7f9a2975cb80 1 Processor -- start 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109cb132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109ca90990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109cb13830 con 0x56109ca93810 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109cac7740 con 0x56109ca91130 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56109ca8f800 con 0x56109ca94120 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a2975cb80 10 client.0 _ll_register_callbacks cb 0x56109c861240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca90450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48330/0 (socket says 172.21.15.83:48330) 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 -- 172.21.15.83:0/183526847 learned_addr learned my addr 172.21.15.83:0/183526847 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109cb132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a177fe700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109ca90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 -- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 msgr2=0x56109ca90990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109ca90990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 -- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 msgr2=0x56109cb132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.944+0000 7f9a167fc700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109cb132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a167fc700 1 -- 172.21.15.83:0/183526847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56109c7e9d90 con 0x56109ca93810 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a177fe700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109ca90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a167fc700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca90450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005590 tx=0x7f9a0c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c98dedd7e3c2383 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 1 -- 172.21.15.83:0/183526847 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c003740 con 0x56109ca93810 2023-11-28T11:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 1 -- 172.21.15.83:0/183526847 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a0c03d070 con 0x56109ca93810 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56109c979160 con 0x56109ca93810 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 1 -- 172.21.15.83:0/183526847 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a0c041470 con 0x56109ca93810 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 10 client.4425 fetch_fsmap learned FSMap version 7 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56109c810200 con 0x56109ca93810 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 1 -- 172.21.15.83:0/183526847 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56109c979160 con 0x56109ca93810 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 1 -- 172.21.15.83:0/183526847 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9a0c0415d0 con 0x56109ca93810 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 10 client.4425 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 10 client.4425 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 20 client.4425 populate_metadata read hostname 'smithi083' 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109caa4af0 0x56109ca68230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 4 client.4425 mds_command: new command op to 4263 tid=0[{"prefix": "status"}] 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 --> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56109c963170 con 0x56109caa4af0 2023-11-28T11:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109caa4af0 0x56109ca68230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.945+0000 7f99f5ffb700 10 client.4425 ms_handle_connect on v2:172.21.15.83:6835/3706545143 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a16ffd700 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109caa4af0 0x56109ca68230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9466c6937d55c84 server_cookie=6d02026f09d53925 in_seq=0 out_seq=0 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f99f5ffb700 1 -- 172.21.15.83:0/183526847 <== mds.0 v2:172.21.15.83:6835/3706545143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56109c963170 con 0x56109caa4af0 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f99f5ffb700 10 client.4425 handle_command_reply: tid=0 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 client.4425 shutdown 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 msgr2=0x56109ca90450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca90450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005590 tx=0x7f9a0c005000 comp rx=0 tx=0).stop 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 shutdown_connections 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x56109caa4af0 0x56109ca68230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56109ca94120 0x56109ca90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f99f5ffb700 0 client.4425 ms_handle_reset on v2:172.21.15.83:6835/3706545143 2023-11-28T11:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56109ca91130 0x56109cb132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 --2- 172.21.15.83:0/183526847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56109ca93810 0x56109ca90450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 >> 172.21.15.83:0/183526847 conn(0x56109ca66d60 msgr2=0x56109ca69530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 shutdown_connections 2023-11-28T11:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 1 -- 172.21.15.83:0/183526847 wait complete. 2023-11-28T11:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a2975cb80 20 client.4425 trim_cache size 0 max 16384 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "69424f07-94c3-4dbc-8507-3551775e0a09", 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4263, 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-28T11:08:41.958 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-28T11:08:41.959 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 6.4270796189999997, 2023-11-28T11:08:41.959 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 7, 2023-11-28T11:08:41.959 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 24, 2023-11-28T11:08:41.959 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 23, 2023-11-28T11:08:41.959 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.992470918 2023-11-28T11:08:41.959 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a040682f0 msgr2=0x7f9a0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a040682f0 0x7f9a0406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a180cd520 tx=0x7f9a0c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 msgr2=0x7f9a18155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.946+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 0x7f9a18155e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c450 tx=0x7f9a0800c810 comp rx=0 tx=0).stop 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 shutdown_connections 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a180d0e40 0x7f9a180ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a040682f0 0x7f9a0406a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a180b4980 0x7f9a18155e60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 --2- 172.21.15.83:0/2196370299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a180b5d50 0x7f9a18152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 >> 172.21.15.83:0/2196370299 conn(0x7f9a1806ceb0 msgr2=0x7f9a180b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 shutdown_connections 2023-11-28T11:08:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:41.947+0000 7f9a1d5d3700 1 -- 172.21.15.83:0/2196370299 wait complete. 2023-11-28T11:08:41.965 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:08:41.965 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:41.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:08:42.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.083+0000 7f27af5f5700 1 Processor -- start 2023-11-28T11:08:42.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.084+0000 7f27af5f5700 1 -- start start 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.084+0000 7f27af5f5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 0x7f27a806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.084+0000 7f27af5f5700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f27a8160410 0x7f27a80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.084+0000 7f27af5f5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a806cab0 con 0x7f27a80ca680 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27af5f5700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a80cf860 con 0x7f27a8160410 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27af5f5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a806b920 con 0x7f27a80b1b10 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27acb90700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f27a80ca680 0x7f27a815fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46198/0 (socket says 172.21.15.83:46198) 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27acb90700 1 -- 172.21.15.83:0/91050199 learned_addr learned my addr 172.21.15.83:0/91050199 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27ad391700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 0x7f27a806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27adb92700 1 --2- 172.21.15.83:0/91050199 >> v2:172.21.15.90:3300/0 conn(0x7f27a8160410 0x7f27a80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183965298 0 0) 0x7f27a4001110 con 0x7f27a80ca680 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27940035b0 con 0x7f27a80ca680 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3035369329 0 0) 0x7f27940035b0 con 0x7f27a80ca680 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 msgr2=0x7f27a806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 0x7f27a806c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 >> v2:172.21.15.90:3300/0 conn(0x7f27a8160410 msgr2=0x7f27a80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 --2- 172.21.15.83:0/91050199 >> v2:172.21.15.90:3300/0 conn(0x7f27a8160410 0x7f27a80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a80cab20 con 0x7f27a80ca680 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f27a4001800 con 0x7f27a80ca680 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 >> v1:172.21.15.83:6789/0 conn(0x7f27a80ca680 legacy=0x7f27a815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2794003a00 0x7f2794003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2794004430 0x7f27940048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 0x7f2794005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80cf860 con 0x7f2794003a00 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a806b920 con 0x7f2794004430 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279400ba10 con 0x7f27a80b1b10 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27ad391700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2794003a00 0x7f2794003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27acb90700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2794004430 0x7f27940048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.085+0000 7f27adb92700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 0x7f2794005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27ad391700 1 -- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 msgr2=0x7f2794005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27ad391700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27a80b1b10 0x7f2794005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27ad391700 1 -- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2794004430 msgr2=0x7f27940048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27ad391700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2794004430 0x7f27940048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27ad391700 1 -- 172.21.15.83:0/91050199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940035b0 con 0x7f2794003a00 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27ad391700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2794003a00 0x7f2794003e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2798009b90 tx=0x7f2794005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67e9566470e0fa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279800e6c0 con 0x7f2794003a00 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279800e820 con 0x7f2794003a00 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f279ffff700 1 -- 172.21.15.83:0/91050199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279800eb90 con 0x7f2794003a00 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27af5f5700 1 -- 172.21.15.83:0/91050199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2794003a00 msgr2=0x7f2794003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27af5f5700 1 --2- 172.21.15.83:0/91050199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2794003a00 0x7f2794003e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2798009b90 tx=0x7f2794005880 comp rx=0 tx=0).stop 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27adb92700 1 -- 172.21.15.83:0/91050199 reap_dead start 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27af5f5700 1 -- 172.21.15.83:0/91050199 shutdown_connections 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27af5f5700 1 -- 172.21.15.83:0/91050199 >> 172.21.15.83:0/91050199 conn(0x7f27a806e9a0 msgr2=0x7f27a80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.086+0000 7f27af5f5700 1 -- 172.21.15.83:0/91050199 shutdown_connections 2023-11-28T11:08:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.087+0000 7f27af5f5700 1 -- 172.21.15.83:0/91050199 wait complete. 2023-11-28T11:08:42.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.088+0000 7f27af5f5700 1 Processor -- start 2023-11-28T11:08:42.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.088+0000 7f27af5f5700 1 -- start start 2023-11-28T11:08:42.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27af5f5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2794003a00 0x7f27a8159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27af5f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a80ce6f0 0x7f27a815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27af5f5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 0x7f27a815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27af5f5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8159af0 con 0x7f27a80ceb40 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27af5f5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81591e0 con 0x7f27a80ce6f0 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27af5f5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a815d050 con 0x7f2794003a00 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 0x7f27a815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 0x7f27a815e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48352/0 (socket says 172.21.15.83:48352) 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 -- 172.21.15.83:0/945120972 learned_addr learned my addr 172.21.15.83:0/945120972 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27acb90700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a80ce6f0 0x7f27a815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27ad391700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2794003a00 0x7f27a8159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 -- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2794003a00 msgr2=0x7f27a8159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2794003a00 0x7f27a8159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 -- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a80ce6f0 msgr2=0x7f27a815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a80ce6f0 0x7f27a815db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27adb92700 1 -- 172.21.15.83:0/945120972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a80cb0e0 con 0x7f27a80ceb40 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.089+0000 7f27ad391700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2794003a00 0x7f27a8159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.090+0000 7f27adb92700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 0x7f27a815e090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27a80b25f0 tx=0x7f27a0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.090+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a000e040 con 0x7f27a80ceb40 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.090+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a000f040 con 0x7f27a80ceb40 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.090+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a80cae00 con 0x7f27a80ceb40 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.090+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a000fd30 con 0x7f27a80ceb40 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.090+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a80cb3c0 con 0x7f27a80ceb40 2023-11-28T11:08:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.091+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a8159c30 con 0x7f27a80ceb40 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.091+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f27a0022460 con 0x7f27a80ceb40 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.092+0000 7f279dffb700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f278c068320 0x7f278c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.092+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f27a0086660 con 0x7f27a80ceb40 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.092+0000 7f27ad391700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f278c068320 0x7f278c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.092+0000 7f27ad391700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f278c068320 0x7f278c06a7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27980090b0 tx=0x7f2798008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.096+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27a8159c30 con 0x7f27a80ceb40 2023-11-28T11:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.322+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f27a8159c30 con 0x7f27a80ceb40 2023-11-28T11:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.322+0000 7f279dffb700 1 -- 172.21.15.83:0/945120972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f27a8159c30 con 0x7f27a80ceb40 2023-11-28T11:08:42.324 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f278c068320 msgr2=0x7f278c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f278c068320 0x7f278c06a7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27980090b0 tx=0x7f2798008030 comp rx=0 tx=0).stop 2023-11-28T11:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 msgr2=0x7f27a815e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 0x7f27a815e090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27a80b25f0 tx=0x7f27a0011ed0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 shutdown_connections 2023-11-28T11:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f278c068320 0x7f278c06a7d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27a80ceb40 0x7f27a815e090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2794003a00 0x7f27a8159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 --2- 172.21.15.83:0/945120972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a80ce6f0 0x7f27a815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 >> 172.21.15.83:0/945120972 conn(0x7f27a806e9a0 msgr2=0x7f27a80b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 shutdown_connections 2023-11-28T11:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:42.324+0000 7f27af5f5700 1 -- 172.21.15.83:0/945120972 wait complete. 2023-11-28T11:08:42.338 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:08:42.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:08:42.338 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-28T11:08:42.338 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-28T11:08:42.338 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:42.338 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:08:42.339 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:08:42.339 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:08:42.339 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T11:08:42.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.339 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-28T11:08:42.356 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42351sec preferred_lft 42351sec 2023-11-28T11:08:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::22f1:9e4c:955b:b4a/64 scope link noprefixroute 2023-11-28T11:08:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:42.358 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:08:42.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.358 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:08:42.358 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:08:42.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:08:42.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2023-11-28T11:08:42.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2023-11-28T11:08:42.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:08:42.359 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:08:42.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.532 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:08:42.560 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:08:42.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.562 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-28T11:08:42.631 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-28T11:08:42.631 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:08:42.631 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:08:42.631 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:08:42.632 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:08:42.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.632 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:08:42.632 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:08:42.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:08:42.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:08:42.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:08:42.633 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:08:42.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.892 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:08:42.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.910 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.968 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:08:42.968 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:08:42.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T11:08:42.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-28T11:08:42.969 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:08:43.063 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-28T11:08:43.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:43.063 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:08:43.063 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:08:43.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-28T11:08:43.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:08:43.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-28T11:08:43.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-28T11:08:43.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-28T11:08:43.064 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:08:43.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:43.249 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:08:43.249 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:08:43.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-28T11:08:43.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:08:43.249 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:08:43.354 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-28T11:08:43.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:43.354 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:43.377 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T11:08:43.377 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:43.377 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:43.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2023-11-28T11:08:43.500 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:08:43.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:08:43.567 INFO:teuthology.orchestra.run.smithi083.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:08:43.568 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:08:43.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:43.568 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2023-11-28T11:08:43.626 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:08:43.626 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_mountpoint=/ --client_fs=cephfs) 2023-11-28T11:08:43.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2023-11-28T11:08:43.694 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:08:43.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:08:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:08:43.760 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:08:43.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:43.761 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2023-11-28T11:08:43.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:2023-11-28T11:08:43.772+0000 7f6beffb65c0 -1 init, newargv = 0x560c72de9d60 newargc=15 2023-11-28T11:08:43.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[110958]: starting ceph client 2023-11-28T11:08:43.779 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[110958]: starting fuse 2023-11-28T11:08:43.790 INFO:teuthology.orchestra.run.smithi083.stdout:47 2023-11-28T11:08:43.790 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> import re 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-28T11:08:43.791 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:08:43.792 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> return f 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.admin")) 2023-11-28T11:08:43.793 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:08:43.913 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.admin.110958.asok 2023-11-28T11:08:44.096 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.110958.asok 2023-11-28T11:08:44.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:44.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.110958.asok status 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "admin", 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-28T11:08:44.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "110958", 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4440, 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2023-11-28T11:08:44.254 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4440 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2120091087 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:08:44.255 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2120091087 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4440 192.168.0.1:0/2120091087", 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2120091087", 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 7, 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 24, 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2023-11-28T11:08:44.256 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:08:44.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:44.263 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2023-11-28T11:08:44.263 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:44.280 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2023-11-28T11:08:44.280 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:44.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:44.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:44.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:44.349 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-28T11:08:44.463 INFO:teuthology.orchestra.run.smithi083.stdout:# file: . 2023-11-28T11:08:44.464 INFO:teuthology.orchestra.run.smithi083.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-28T11:08:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:44.464 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:44.464 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:44.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.640+0000 7f3b22e79700 1 Processor -- start 2023-11-28T11:08:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b22e79700 1 -- start start 2023-11-28T11:08:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b22e79700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3b1c06d100 0x7f3b1c160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b22e79700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b1c0cd070 0x7f3b1c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b22e79700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c160d40 con 0x7f3b1c0cadc0 2023-11-28T11:08:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b22e79700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c161020 con 0x7f3b1c0cd070 2023-11-28T11:08:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b22e79700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c161300 con 0x7f3b1c06d100 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b1bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b1c0cd070 0x7f3b1c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b1bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b1c0cd070 0x7f3b1c06b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48374/0 (socket says 172.21.15.83:48374) 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.641+0000 7f3b1bfff700 1 -- 172.21.15.83:0/1464505586 learned_addr learned my addr 172.21.15.83:0/1464505586 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b20c15700 1 --2- 172.21.15.83:0/1464505586 >> v2:172.21.15.90:3300/0 conn(0x7f3b1c06d100 0x7f3b1c160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1bfff700 1 -- 172.21.15.83:0/1464505586 >> v2:172.21.15.90:3300/0 conn(0x7f3b1c06d100 msgr2=0x7f3b1c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1bfff700 1 --2- 172.21.15.83:0/1464505586 >> v2:172.21.15.90:3300/0 conn(0x7f3b1c06d100 0x7f3b1c160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1bfff700 1 -- 172.21.15.83:0/1464505586 >> v1:172.21.15.90:6789/0 conn(0x7f3b1c0cadc0 legacy=0x7f3b1c0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1bfff700 1 -- 172.21.15.83:0/1464505586 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0ce900 con 0x7f3b1c0cd070 2023-11-28T11:08:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1bfff700 1 --2- 172.21.15.83:0/1464505586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c0cd070 0x7f3b1c06b260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0cd930 tx=0x7f3b08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7585490f2d2124b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1b7fe700 1 -- 172.21.15.83:0/1464505586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08004230 con 0x7f3b1c0cd070 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1b7fe700 1 -- 172.21.15.83:0/1464505586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b08004390 con 0x7f3b1c0cd070 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b1b7fe700 1 -- 172.21.15.83:0/1464505586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080036a0 con 0x7f3b1c0cd070 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b22e79700 1 -- 172.21.15.83:0/1464505586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c0cd070 msgr2=0x7f3b1c06b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.642+0000 7f3b22e79700 1 --2- 172.21.15.83:0/1464505586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c0cd070 0x7f3b1c06b260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0cd930 tx=0x7f3b08009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.643+0000 7f3b22e79700 1 -- 172.21.15.83:0/1464505586 shutdown_connections 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.643+0000 7f3b22e79700 1 --2- 172.21.15.83:0/1464505586 >> v2:172.21.15.90:3300/0 conn(0x7f3b1c06d100 0x7f3b1c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.643+0000 7f3b22e79700 1 --2- 172.21.15.83:0/1464505586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c0cd070 0x7f3b1c06b260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.643+0000 7f3b22e79700 1 -- 172.21.15.83:0/1464505586 >> 172.21.15.83:0/1464505586 conn(0x7f3b1c06dd40 msgr2=0x7f3b1c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.643+0000 7f3b22e79700 1 -- 172.21.15.83:0/1464505586 shutdown_connections 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.643+0000 7f3b22e79700 1 -- 172.21.15.83:0/1464505586 wait complete. 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.644+0000 7f3b22e79700 1 Processor -- start 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.644+0000 7f3b22e79700 1 -- start start 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 0x7f3b1c1fa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b1c0cadc0 0x7f3b1c0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b1c0cd070 0x7f3b1c1f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0b1140 con 0x7f3b1c06d100 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1f7750 con 0x7f3b1c0cadc0 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1fa2b0 con 0x7f3b1c0cd070 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 0x7f3b1c1fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 0x7f3b1c1fa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48378/0 (socket says 172.21.15.83:48378) 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 -- 172.21.15.83:0/67230386 learned_addr learned my addr 172.21.15.83:0/67230386 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b1bfff700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b1c0cadc0 0x7f3b1c0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b21416700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b1c0cd070 0x7f3b1c1f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 -- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b1c0cd070 msgr2=0x7f3b1c1f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b1c0cd070 0x7f3b1c1f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 -- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b1c0cadc0 msgr2=0x7f3b1c0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b1c0cadc0 0x7f3b1c0c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 -- 172.21.15.83:0/67230386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009550 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b20c15700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 0x7f3b1c1fa740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000e810 tx=0x7f3b1000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10013610 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10013770 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c160980 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10013b20 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.645+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c0cebe0 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.646+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c160b10 con 0x7f3b1c06d100 2023-11-28T11:08:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.647+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b10023070 con 0x7f3b1c06d100 2023-11-28T11:08:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.647+0000 7f3b197fa700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3b00068320 0x7f3b0006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.647+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3b10085be0 con 0x7f3b1c06d100 2023-11-28T11:08:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.647+0000 7f3b1bfff700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3b00068320 0x7f3b0006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.647+0000 7f3b1bfff700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3b00068320 0x7f3b0006a7d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0cd470 tx=0x7f3b08008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.651+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b100520d0 con 0x7f3b1c06d100 2023-11-28T11:08:44.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.836+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b1c1fa590 con 0x7f3b1c06d100 2023-11-28T11:08:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.838+0000 7f3b197fa700 1 -- 172.21.15.83:0/67230386 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f3b100556f0 con 0x7f3b1c06d100 2023-11-28T11:08:44.842 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:44.843 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3b00068320 msgr2=0x7f3b0006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3b00068320 0x7f3b0006a7d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0cd470 tx=0x7f3b08008030 comp rx=0 tx=0).stop 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 msgr2=0x7f3b1c1fa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 0x7f3b1c1fa740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000e810 tx=0x7f3b1000eb20 comp rx=0 tx=0).stop 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 shutdown_connections 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b1c0cd070 0x7f3b1c1f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3b00068320 0x7f3b0006a7d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c06d100 0x7f3b1c1fa740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 --2- 172.21.15.83:0/67230386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b1c0cadc0 0x7f3b1c0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 >> 172.21.15.83:0/67230386 conn(0x7f3b1c06dd40 msgr2=0x7f3b1c0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 shutdown_connections 2023-11-28T11:08:44.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.840+0000 7f3b22e79700 1 -- 172.21.15.83:0/67230386 wait complete. 2023-11-28T11:08:44.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:08:44.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.975+0000 7f110f89b700 1 Processor -- start 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.975+0000 7f110f89b700 1 -- start start 2023-11-28T11:08:44.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110f89b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11080aee60 0x7f11080b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110f89b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110806d4a0 con 0x7f110806e370 2023-11-28T11:08:44.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110f89b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11080b5ae0 con 0x7f110815c890 2023-11-28T11:08:44.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110f89b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110806c400 con 0x7f11080aee60 2023-11-28T11:08:44.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110de38700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11080aee60 0x7f11080b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110ce36700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f110806e370 0x7f110806b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46246/0 (socket says 172.21.15.83:46246) 2023-11-28T11:08:44.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.976+0000 7f110ce36700 1 -- 172.21.15.83:0/3249347390 learned_addr learned my addr 172.21.15.83:0/3249347390 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:44.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f110de38700 1 -- 172.21.15.83:0/3249347390 >> v1:172.21.15.90:6789/0 conn(0x7f110815c890 legacy=0x7f110806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f110de38700 1 -- 172.21.15.83:0/3249347390 >> v1:172.21.15.83:6789/0 conn(0x7f110806e370 legacy=0x7f110806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f10fffff700 1 -- 172.21.15.83:0/3249347390 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091405489 0 0) 0x7f110806d4a0 con 0x7f110806e370 2023-11-28T11:08:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f110de38700 1 -- 172.21.15.83:0/3249347390 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080b5240 con 0x7f11080aee60 2023-11-28T11:08:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f110de38700 1 --2- 172.21.15.83:0/3249347390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080b4bb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1100009d90 tx=0x7f110000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33965d2d5b25efde server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f10fffff700 1 -- 172.21.15.83:0/3249347390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110000e070 con 0x7f11080aee60 2023-11-28T11:08:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f10fffff700 1 -- 172.21.15.83:0/3249347390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110000bbd0 con 0x7f11080aee60 2023-11-28T11:08:44.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f10fffff700 1 -- 172.21.15.83:0/3249347390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100012430 con 0x7f11080aee60 2023-11-28T11:08:44.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f110f89b700 1 -- 172.21.15.83:0/3249347390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 msgr2=0x7f11080b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.977+0000 7f110f89b700 1 --2- 172.21.15.83:0/3249347390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080b4bb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1100009d90 tx=0x7f110000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:44.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.978+0000 7f110f89b700 1 -- 172.21.15.83:0/3249347390 shutdown_connections 2023-11-28T11:08:44.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.978+0000 7f110f89b700 1 --2- 172.21.15.83:0/3249347390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080b4bb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.978+0000 7f110f89b700 1 -- 172.21.15.83:0/3249347390 >> 172.21.15.83:0/3249347390 conn(0x7f11080b2b60 msgr2=0x7f11080b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.978+0000 7f110f89b700 1 -- 172.21.15.83:0/3249347390 shutdown_connections 2023-11-28T11:08:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.978+0000 7f110f89b700 1 -- 172.21.15.83:0/3249347390 wait complete. 2023-11-28T11:08:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.979+0000 7f110f89b700 1 Processor -- start 2023-11-28T11:08:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.979+0000 7f110f89b700 1 -- start start 2023-11-28T11:08:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.979+0000 7f110f89b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110806e370 0x7f11080c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.979+0000 7f110f89b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110f89b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110815c890 0x7f11081f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110f89b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c6250 con 0x7f11080aee60 2023-11-28T11:08:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110f89b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081f2070 con 0x7f110815c890 2023-11-28T11:08:44.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110f89b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c4790 con 0x7f110806e370 2023-11-28T11:08:44.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48394/0 (socket says 172.21.15.83:48394) 2023-11-28T11:08:44.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 -- 172.21.15.83:0/1348242806 learned_addr learned my addr 172.21.15.83:0/1348242806 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:44.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110de38700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110815c890 0x7f11081f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110d637700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110806e370 0x7f11080c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 -- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110806e370 msgr2=0x7f11080c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110806e370 0x7f11080c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 -- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110815c890 msgr2=0x7f11081f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110815c890 0x7f11081f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 -- 172.21.15.83:0/1348242806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100009a70 con 0x7f11080aee60 2023-11-28T11:08:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110d637700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110806e370 0x7f11080c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110ce36700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080c5d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f110400c450 tx=0x7f110400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104004c80 con 0x7f11080aee60 2023-11-28T11:08:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104004e00 con 0x7f11080aee60 2023-11-28T11:08:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11080b5800 con 0x7f11080aee60 2023-11-28T11:08:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104007310 con 0x7f11080aee60 2023-11-28T11:08:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.980+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11080b5520 con 0x7f11080aee60 2023-11-28T11:08:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.981+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110806ac30 con 0x7f11080aee60 2023-11-28T11:08:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.982+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f110401a020 con 0x7f11080aee60 2023-11-28T11:08:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.982+0000 7f10fdffb700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10ec0682f0 0x7f10ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.982+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1104085aa0 con 0x7f11080aee60 2023-11-28T11:08:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.982+0000 7f110d637700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10ec0682f0 0x7f10ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.983+0000 7f110d637700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10ec0682f0 0x7f10ec06a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10f8000fb0 tx=0x7f10f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:44.986+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1104051f80 con 0x7f11080aee60 2023-11-28T11:08:45.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.170+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f11080b4bb0 con 0x7f11080aee60 2023-11-28T11:08:45.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.171+0000 7f10fdffb700 1 -- 172.21.15.83:0/1348242806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f11040555a0 con 0x7f11080aee60 2023-11-28T11:08:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:08:45.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10ec0682f0 msgr2=0x7f10ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10ec0682f0 0x7f10ec06a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10f8000fb0 tx=0x7f10f8009040 comp rx=0 tx=0).stop 2023-11-28T11:08:45.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 msgr2=0x7f11080c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080c5d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f110400c450 tx=0x7f110400c760 comp rx=0 tx=0).stop 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 shutdown_connections 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110815c890 0x7f11081f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110806e370 0x7f11080c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10ec0682f0 0x7f10ec06a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 --2- 172.21.15.83:0/1348242806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080aee60 0x7f11080c5d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 >> 172.21.15.83:0/1348242806 conn(0x7f11080b2b60 msgr2=0x7f110806b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 shutdown_connections 2023-11-28T11:08:45.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.173+0000 7f110f89b700 1 -- 172.21.15.83:0/1348242806 wait complete. 2023-11-28T11:08:45.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0ba2474700 1 Processor -- start 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0ba2474700 1 -- start start 2023-11-28T11:08:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0ba2474700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b9c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0ba2474700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c06d4a0 con 0x7f0b9c15c890 2023-11-28T11:08:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0ba2474700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c0b5ae0 con 0x7f0b9c06e370 2023-11-28T11:08:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0ba2474700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c06c400 con 0x7f0b9c0aee60 2023-11-28T11:08:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.309+0000 7f0b9bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0b9c15c890 0x7f0b9c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46260/0 (socket says 172.21.15.83:46260) 2023-11-28T11:08:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 -- 172.21.15.83:0/1570811578 learned_addr learned my addr 172.21.15.83:0/1570811578 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0ba0a11700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b9c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525795228 0 0) 0x7f0b9c06d4a0 con 0x7f0b9c15c890 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b840035b0 con 0x7f0b9c15c890 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61099058 0 0) 0x7f0b9c0b5ae0 con 0x7f0b9c06e370 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b9c06d4a0 con 0x7f0b9c06e370 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3217259266 0 0) 0x7f0b840035b0 con 0x7f0b9c15c890 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 msgr2=0x7f0b9c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b9c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 >> v1:172.21.15.90:6789/0 conn(0x7f0b9c06e370 legacy=0x7f0b9c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0b5240 con 0x7f0b9c15c890 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f0b8c0017f0 con 0x7f0b9c15c890 2023-11-28T11:08:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0ba0a11700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b9c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 >> v1:172.21.15.83:6789/0 conn(0x7f0b9c15c890 legacy=0x7f0b9c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b84003bc0 0x7f0b84003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84004850 0x7f0b84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b84005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c06c400 con 0x7f0b84003bc0 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8400baf0 con 0x7f0b84004850 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8400bcb0 con 0x7f0b9c0aee60 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b84003bc0 0x7f0b84003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9b7fe700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84004850 0x7f0b84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0ba0a11700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b84005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 -- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 msgr2=0x7f0b84005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0aee60 0x7f0b84005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 -- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84004850 msgr2=0x7f0b84004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84004850 0x7f0b84004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.310+0000 7f0b9bfff700 1 -- 172.21.15.83:0/1570811578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94009a70 con 0x7f0b84003bc0 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b84003bc0 0x7f0b84003fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c240 tx=0x7f0b84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e256c611d47fbbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c00cd80 con 0x7f0b84003bc0 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c007b70 con 0x7f0b84003bc0 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0b9affd700 1 -- 172.21.15.83:0/1570811578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c007310 con 0x7f0b84003bc0 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba2474700 1 -- 172.21.15.83:0/1570811578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b84003bc0 msgr2=0x7f0b84003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba2474700 1 --2- 172.21.15.83:0/1570811578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b84003bc0 0x7f0b84003fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c240 tx=0x7f0b84005880 comp rx=0 tx=0).stop 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba0a11700 1 -- 172.21.15.83:0/1570811578 reap_dead start 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba2474700 1 -- 172.21.15.83:0/1570811578 shutdown_connections 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba2474700 1 -- 172.21.15.83:0/1570811578 >> 172.21.15.83:0/1570811578 conn(0x7f0b9c0b2b60 msgr2=0x7f0b9c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba2474700 1 -- 172.21.15.83:0/1570811578 shutdown_connections 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.311+0000 7f0ba2474700 1 -- 172.21.15.83:0/1570811578 wait complete. 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.312+0000 7f0ba2474700 1 Processor -- start 2023-11-28T11:08:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 -- start start 2023-11-28T11:08:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84003bc0 0x7f0b9c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 0x7f0b9c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0c8cf0 0x7f0b9c0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c54b0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c96a0 con 0x7f0b84003bc0 2023-11-28T11:08:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.313+0000 7f0ba2474700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c4830 con 0x7f0b9c0c8cf0 2023-11-28T11:08:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 0x7f0b9c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 0x7f0b9c0c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48412/0 (socket says 172.21.15.83:48412) 2023-11-28T11:08:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 -- 172.21.15.83:0/2032218214 learned_addr learned my addr 172.21.15.83:0/2032218214 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84003bc0 0x7f0b9c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0ba0a11700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0c8cf0 0x7f0b9c0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 -- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0c8cf0 msgr2=0x7f0b9c0c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0c8cf0 0x7f0b9c0c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 -- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84003bc0 msgr2=0x7f0b9c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84003bc0 0x7f0b9c158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 -- 172.21.15.83:0/2032218214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0b5800 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b9b7fe700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 0x7f0b9c0c6020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000f7a0 tx=0x7f0b9000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b900053e0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90005540 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90006aa0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c0b5520 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.314+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c06ac30 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.315+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c0c55f0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.316+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b90006c00 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.316+0000 7f0b98ff9700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0b880682f0 0x7f0b8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.316+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0b90085580 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.316+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0b880682f0 0x7f0b8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.317+0000 7f0b9bfff700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0b880682f0 0x7f0b8806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c015040 tx=0x7f0b8c00a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.320+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0b9c0c55f0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.502+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b9c0c55f0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.503+0000 7f0b98ff9700 1 -- 172.21.15.83:0/2032218214 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b9c0c55f0 con 0x7f0b9c0c5bb0 2023-11-28T11:08:45.504 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0b880682f0 msgr2=0x7f0b8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0b880682f0 0x7f0b8806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c015040 tx=0x7f0b8c00a000 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 msgr2=0x7f0b9c0c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 0x7f0b9c0c6020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000f7a0 tx=0x7f0b9000fab0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 shutdown_connections 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0b9c0c8cf0 0x7f0b9c0c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b84003bc0 0x7f0b9c158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0b880682f0 0x7f0b8806a7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.504+0000 7f0ba2474700 1 --2- 172.21.15.83:0/2032218214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b9c0c5bb0 0x7f0b9c0c6020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.505+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 >> 172.21.15.83:0/2032218214 conn(0x7f0b9c0b2b60 msgr2=0x7f0b9c0aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.505+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 shutdown_connections 2023-11-28T11:08:45.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.505+0000 7f0ba2474700 1 -- 172.21.15.83:0/2032218214 wait complete. 2023-11-28T11:08:45.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:45.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 Processor -- start 2023-11-28T11:08:45.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 -- start start 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80b00aee20 0x7f80b00b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b00cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b00b5b00 con 0x7f80b006c370 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b006c800 con 0x7f80b00aee20 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80b5156700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b006cae0 con 0x7f80b00af1f0 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.639+0000 7f80aed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80b00aee20 0x7f80b00b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80b00aee20 0x7f80b00b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48420/0 (socket says 172.21.15.83:48420) 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80ae59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f80b006c370 0x7f80b006ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46280/0 (socket says 172.21.15.83:46280) 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 -- 172.21.15.83:0/1189949442 learned_addr learned my addr 172.21.15.83:0/1189949442 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80af59e700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b00cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 -- 172.21.15.83:0/1189949442 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 msgr2=0x7f80b00cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b00cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 -- 172.21.15.83:0/1189949442 >> v1:172.21.15.83:6789/0 conn(0x7f80b006c370 legacy=0x7f80b006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 -- 172.21.15.83:0/1189949442 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b006cdc0 con 0x7f80b00aee20 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80add9b700 1 -- 172.21.15.83:0/1189949442 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922986162 0 0) 0x7f80a0001110 con 0x7f80b006c370 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80af59e700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b00cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80aed9d700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b00aee20 0x7f80b00b55c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006fd0 tx=0x7f80a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cab0a15cfd1ade server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80add9b700 1 -- 172.21.15.83:0/1189949442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4005e30 con 0x7f80b00aee20 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80add9b700 1 -- 172.21.15.83:0/1189949442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4004030 con 0x7f80b00aee20 2023-11-28T11:08:45.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.640+0000 7f80add9b700 1 -- 172.21.15.83:0/1189949442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40043e0 con 0x7f80b00aee20 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 -- 172.21.15.83:0/1189949442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b00aee20 msgr2=0x7f80b00b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b00aee20 0x7f80b00b55c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006fd0 tx=0x7f80a4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 -- 172.21.15.83:0/1189949442 shutdown_connections 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b00cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 --2- 172.21.15.83:0/1189949442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b00aee20 0x7f80b00b55c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 -- 172.21.15.83:0/1189949442 >> 172.21.15.83:0/1189949442 conn(0x7f80b00b2b60 msgr2=0x7f80b00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 -- 172.21.15.83:0/1189949442 shutdown_connections 2023-11-28T11:08:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.641+0000 7f80b5156700 1 -- 172.21.15.83:0/1189949442 wait complete. 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.642+0000 7f80b5156700 1 Processor -- start 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.642+0000 7f80b5156700 1 -- start start 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80b5156700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 0x7f80b00b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80b5156700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b00aee20 0x7f80b00af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80b5156700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b01f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80b5156700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00b0660 con 0x7f80b006c370 2023-11-28T11:08:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80b5156700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00afbb0 con 0x7f80b00aee20 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80b5156700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00afdc0 con 0x7f80b00af1f0 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 0x7f80b00b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 0x7f80b00b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48426/0 (socket says 172.21.15.83:48426) 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 -- 172.21.15.83:0/15031011 learned_addr learned my addr 172.21.15.83:0/15031011 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80af59e700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b01f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80ae59c700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b00aee20 0x7f80b00af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 -- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 msgr2=0x7f80b01f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b01f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 -- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b00aee20 msgr2=0x7f80b00af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b00aee20 0x7f80b00af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.643+0000 7f80aed9d700 1 -- 172.21.15.83:0/15031011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4009500 con 0x7f80b006c370 2023-11-28T11:08:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.644+0000 7f80aed9d700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 0x7f80b00b1570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009c10 tx=0x7f80a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.644+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4004560 con 0x7f80b006c370 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.644+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4011420 con 0x7f80b006c370 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.644+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b006fc70 con 0x7f80b006c370 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.644+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40117d0 con 0x7f80b006c370 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.644+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b006d0a0 con 0x7f80b006c370 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.645+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b006d380 con 0x7f80b006c370 2023-11-28T11:08:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.645+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80a402b070 con 0x7f80b006c370 2023-11-28T11:08:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.645+0000 7f80977fe700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f809c0682f0 0x7f809c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.645+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f80a4086e20 con 0x7f80b006c370 2023-11-28T11:08:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.645+0000 7f80ae59c700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f809c0682f0 0x7f809c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.646+0000 7f80ae59c700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f809c0682f0 0x7f809c06a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f80b00b20b0 tx=0x7f80a0007c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.650+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80a4053390 con 0x7f80b006c370 2023-11-28T11:08:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.834+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80b00c66f0 con 0x7f80b006c370 2023-11-28T11:08:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.834+0000 7f80977fe700 1 -- 172.21.15.83:0/15031011 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80a40569b0 con 0x7f80b006c370 2023-11-28T11:08:45.835 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:08:45.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f809c0682f0 msgr2=0x7f809c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f809c0682f0 0x7f809c06a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f80b00b20b0 tx=0x7f80a0007c20 comp rx=0 tx=0).stop 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 msgr2=0x7f80b00b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 0x7f80b00b1570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009c10 tx=0x7f80a4004750 comp rx=0 tx=0).stop 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 shutdown_connections 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80b00af1f0 0x7f80b01f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f809c0682f0 0x7f809c06a7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80b006c370 0x7f80b00b1570 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 --2- 172.21.15.83:0/15031011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80b00aee20 0x7f80b00af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 >> 172.21.15.83:0/15031011 conn(0x7f80b00b2b60 msgr2=0x7f80b006ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 shutdown_connections 2023-11-28T11:08:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.836+0000 7f80b5156700 1 -- 172.21.15.83:0/15031011 wait complete. 2023-11-28T11:08:45.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.970+0000 7fe0cd333700 1 Processor -- start 2023-11-28T11:08:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0cd333700 1 -- start start 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0cd333700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80d0dc0 0x7fe0c815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0cd333700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0c815d1a0 0x7fe0c806c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0cd333700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c8157140 con 0x7fe0c8156d70 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0cd333700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c815dcc0 con 0x7fe0c815d1a0 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0cd333700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c815d9e0 con 0x7fe0c80d0dc0 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0c77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0c815d1a0 0x7fe0c806c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0c77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0c815d1a0 0x7fe0c806c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48438/0 (socket says 172.21.15.83:48438) 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.971+0000 7fe0c77fe700 1 -- 172.21.15.83:0/4259150976 learned_addr learned my addr 172.21.15.83:0/4259150976 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c67fc700 1 --2- 172.21.15.83:0/4259150976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80d0dc0 0x7fe0c815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c77fe700 1 -- 172.21.15.83:0/4259150976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80d0dc0 msgr2=0x7fe0c815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c77fe700 1 --2- 172.21.15.83:0/4259150976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80d0dc0 0x7fe0c815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c77fe700 1 -- 172.21.15.83:0/4259150976 >> v1:172.21.15.90:6789/0 conn(0x7fe0c8156d70 legacy=0x7fe0c80d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c77fe700 1 -- 172.21.15.83:0/4259150976 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c806bdb0 con 0x7fe0c815d1a0 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c5ffb700 1 -- 172.21.15.83:0/4259150976 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457500665 0 0) 0x7fe0c8157140 con 0x7fe0c8156d70 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c77fe700 1 --2- 172.21.15.83:0/4259150976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c815d1a0 0x7fe0c806c890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009e50 tx=0x7fe0c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50014cd7cab4171a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c5ffb700 1 -- 172.21.15.83:0/4259150976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000e070 con 0x7fe0c815d1a0 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c5ffb700 1 -- 172.21.15.83:0/4259150976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c000b910 con 0x7fe0c815d1a0 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.972+0000 7fe0c5ffb700 1 -- 172.21.15.83:0/4259150976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000bcc0 con 0x7fe0c815d1a0 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 -- 172.21.15.83:0/4259150976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c815d1a0 msgr2=0x7fe0c806c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 --2- 172.21.15.83:0/4259150976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c815d1a0 0x7fe0c806c890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009e50 tx=0x7fe0c000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 -- 172.21.15.83:0/4259150976 shutdown_connections 2023-11-28T11:08:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 --2- 172.21.15.83:0/4259150976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c815d1a0 0x7fe0c806c890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 --2- 172.21.15.83:0/4259150976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80d0dc0 0x7fe0c815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 -- 172.21.15.83:0/4259150976 >> 172.21.15.83:0/4259150976 conn(0x7fe0c806ea50 msgr2=0x7fe0c8070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 -- 172.21.15.83:0/4259150976 shutdown_connections 2023-11-28T11:08:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.973+0000 7fe0cd333700 1 -- 172.21.15.83:0/4259150976 wait complete. 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.974+0000 7fe0cd333700 1 Processor -- start 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.974+0000 7fe0cd333700 1 -- start start 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 0x7fe0c80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c815d1a0 0x7fe0c80ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80c9550 0x7fe0c814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80cd080 con 0x7fe0c8156d70 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80c84b0 con 0x7fe0c815d1a0 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80c8790 con 0x7fe0c80c9550 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 0x7fe0c80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 0x7fe0c80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48440/0 (socket says 172.21.15.83:48440) 2023-11-28T11:08:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 -- 172.21.15.83:0/2024489425 learned_addr learned my addr 172.21.15.83:0/2024489425 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c67fc700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c815d1a0 0x7fe0c80ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c77fe700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80c9550 0x7fe0c814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 -- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80c9550 msgr2=0x7fe0c814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80c9550 0x7fe0c814ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 -- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c815d1a0 msgr2=0x7fe0c80ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c815d1a0 0x7fe0c80ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 -- 172.21.15.83:0/2024489425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0009b30 con 0x7fe0c8156d70 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0c6ffd700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 0x7fe0c80c7db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0011e0 tx=0x7fe0bc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0054e0 con 0x7fe0c8156d70 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc003680 con 0x7fe0c8156d70 2023-11-28T11:08:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c806c370 con 0x7fe0c8156d70 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc003a30 con 0x7fe0c8156d70 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.975+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c806c090 con 0x7fe0c8156d70 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.976+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c80cf1e0 con 0x7fe0c8156d70 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.977+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0bc019030 con 0x7fe0c8156d70 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.977+0000 7fe0af7fe700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe0b40682f0 0x7fe0b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.977+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe0bc085270 con 0x7fe0c8156d70 2023-11-28T11:08:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.977+0000 7fe0c67fc700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe0b40682f0 0x7fe0b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.978+0000 7fe0c67fc700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe0b40682f0 0x7fe0b406a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80d0460 tx=0x7fe0b8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:45.981+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0bc0517e0 con 0x7fe0c8156d70 2023-11-28T11:08:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.162+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0c80742b0 con 0x7fe0c8156d70 2023-11-28T11:08:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.163+0000 7fe0af7fe700 1 -- 172.21.15.83:0/2024489425 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0bc017020 con 0x7fe0c8156d70 2023-11-28T11:08:46.164 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:08:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.164+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe0b40682f0 msgr2=0x7fe0b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.164+0000 7fe0cd333700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe0b40682f0 0x7fe0b406a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80d0460 tx=0x7fe0b8009210 comp rx=0 tx=0).stop 2023-11-28T11:08:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.164+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 msgr2=0x7fe0c80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.164+0000 7fe0cd333700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 0x7fe0c80c7db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0011e0 tx=0x7fe0bc00ca30 comp rx=0 tx=0).stop 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 shutdown_connections 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c80c9550 0x7fe0c814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe0b40682f0 0x7fe0b406a7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0c8156d70 0x7fe0c80c7db0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 --2- 172.21.15.83:0/2024489425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c815d1a0 0x7fe0c80ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 >> 172.21.15.83:0/2024489425 conn(0x7fe0c80d0dc0 msgr2=0x7fe0c80b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 shutdown_connections 2023-11-28T11:08:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.165+0000 7fe0cd333700 1 -- 172.21.15.83:0/2024489425 wait complete. 2023-11-28T11:08:46.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:08:46.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.299+0000 7f505e4bc700 1 Processor -- start 2023-11-28T11:08:46.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.299+0000 7f505e4bc700 1 -- start start 2023-11-28T11:08:46.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f505e4bc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50580ce1b0 0x7f505815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f505e4bc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580d0ae0 0x7f50580d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f505e4bc700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5058151a30 con 0x7f505815e5c0 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f505e4bc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50580cb460 con 0x7f50580ce1b0 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f505e4bc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50580cb740 con 0x7f50580d0ae0 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50580ce1b0 0x7f505815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50580ce1b0 0x7f505815b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48450/0 (socket says 172.21.15.83:48450) 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 -- 172.21.15.83:0/211845570 learned_addr learned my addr 172.21.15.83:0/211845570 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f505ca59700 1 --2- 172.21.15.83:0/211845570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580d0ae0 0x7f50580d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 -- 172.21.15.83:0/211845570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580d0ae0 msgr2=0x7f50580d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 --2- 172.21.15.83:0/211845570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580d0ae0 0x7f50580d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 -- 172.21.15.83:0/211845570 >> v1:172.21.15.90:6789/0 conn(0x7f505815e5c0 legacy=0x7f505815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.300+0000 7f50577fe700 1 -- 172.21.15.83:0/211845570 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580d1480 con 0x7f50580ce1b0 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f50577fe700 1 --2- 172.21.15.83:0/211845570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580ce1b0 0x7f505815b3b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50480098f0 tx=0x7f5048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea52ba7e915c668f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f5056ffd700 1 -- 172.21.15.83:0/211845570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048011830 con 0x7f50580ce1b0 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f5056ffd700 1 -- 172.21.15.83:0/211845570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5048011990 con 0x7f50580ce1b0 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f5056ffd700 1 -- 172.21.15.83:0/211845570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048011d20 con 0x7f50580ce1b0 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 -- 172.21.15.83:0/211845570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580ce1b0 msgr2=0x7f505815b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 --2- 172.21.15.83:0/211845570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580ce1b0 0x7f505815b3b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50480098f0 tx=0x7f5048009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 -- 172.21.15.83:0/211845570 shutdown_connections 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 --2- 172.21.15.83:0/211845570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580d0ae0 0x7f50580d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 --2- 172.21.15.83:0/211845570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580ce1b0 0x7f505815b3b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 -- 172.21.15.83:0/211845570 >> 172.21.15.83:0/211845570 conn(0x7f505806ea50 msgr2=0x7f5058070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 -- 172.21.15.83:0/211845570 shutdown_connections 2023-11-28T11:08:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.301+0000 7f505e4bc700 1 -- 172.21.15.83:0/211845570 wait complete. 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 Processor -- start 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 -- start start 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 0x7f505814dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505815e5c0 0x7f50580ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50580caa20 0x7f50580b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580b3690 con 0x7f50580d0ae0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50581f0390 con 0x7f50580caa20 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.303+0000 7f505e4bc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50580b2a80 con 0x7f505815e5c0 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 0x7f505814dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 0x7f505814dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48454/0 (socket says 172.21.15.83:48454) 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 -- 172.21.15.83:0/3525756978 learned_addr learned my addr 172.21.15.83:0/3525756978 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f50577fe700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505815e5c0 0x7f50580ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f505ca59700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50580caa20 0x7f50580b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 -- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505815e5c0 msgr2=0x7f50580ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505815e5c0 0x7f50580ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 -- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50580caa20 msgr2=0x7f50580b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50580caa20 0x7f50580b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 -- 172.21.15.83:0/3525756978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048009550 con 0x7f50580d0ae0 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f50577fe700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505815e5c0 0x7f50580ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f505ca59700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50580caa20 0x7f50580b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5057fff700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 0x7f505814dd00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f504c00c510 tx=0x7f504c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0052d0 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c005430 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505815ae90 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0038c0 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.304+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505815abb0 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.305+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505815b8f0 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.306+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f504c019030 con 0x7f50580d0ae0 2023-11-28T11:08:46.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.306+0000 7f5054ff9700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50440682f0 0x7f504406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.306+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f504c085330 con 0x7f50580d0ae0 2023-11-28T11:08:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.306+0000 7f50577fe700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50440682f0 0x7f504406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.306+0000 7f50577fe700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50440682f0 0x7f504406a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5058159cf0 tx=0x7f5048008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.310+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f504c0518a0 con 0x7f50580d0ae0 2023-11-28T11:08:46.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.492+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f50580742b0 con 0x7f50580d0ae0 2023-11-28T11:08:46.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.492+0000 7f5054ff9700 1 -- 172.21.15.83:0/3525756978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f504c054ec0 con 0x7f50580d0ae0 2023-11-28T11:08:46.494 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50440682f0 msgr2=0x7f504406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50440682f0 0x7f504406a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5058159cf0 tx=0x7f5048008030 comp rx=0 tx=0).stop 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 msgr2=0x7f505814dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 0x7f505814dd00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f504c00c510 tx=0x7f504c00c8d0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 shutdown_connections 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50580caa20 0x7f50580b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50440682f0 0x7f504406a7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50580d0ae0 0x7f505814dd00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 --2- 172.21.15.83:0/3525756978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505815e5c0 0x7f50580ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 >> 172.21.15.83:0/3525756978 conn(0x7f50580ce1b0 msgr2=0x7f505815d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 shutdown_connections 2023-11-28T11:08:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.494+0000 7f505e4bc700 1 -- 172.21.15.83:0/3525756978 wait complete. 2023-11-28T11:08:46.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:08:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.626+0000 7fc5fd72a700 1 Processor -- start 2023-11-28T11:08:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.626+0000 7fc5fd72a700 1 -- start start 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5fd72a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc5f80ceee0 0x7fc5f8160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5fd72a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5f81607c0 0x7fc5f80d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5fd72a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f80d1780 con 0x7fc5f80d0cb0 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5fd72a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f8160c40 con 0x7fc5f81607c0 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5fd72a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f8161140 con 0x7fc5f80ceee0 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5f81607c0 0x7fc5f80d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5f81607c0 0x7fc5f80d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48462/0 (socket says 172.21.15.83:48462) 2023-11-28T11:08:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc5f80d0cb0 0x7fc5f815cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46324/0 (socket says 172.21.15.83:46324) 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 -- 172.21.15.83:0/169805128 learned_addr learned my addr 172.21.15.83:0/169805128 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f6ffd700 1 --2- 172.21.15.83:0/169805128 >> v2:172.21.15.90:3300/0 conn(0x7fc5f80ceee0 0x7fc5f8160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 -- 172.21.15.83:0/169805128 >> v2:172.21.15.90:3300/0 conn(0x7fc5f80ceee0 msgr2=0x7fc5f8160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 --2- 172.21.15.83:0/169805128 >> v2:172.21.15.90:3300/0 conn(0x7fc5f80ceee0 0x7fc5f8160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 -- 172.21.15.83:0/169805128 >> v1:172.21.15.83:6789/0 conn(0x7fc5f80d0cb0 legacy=0x7fc5f815cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 -- 172.21.15.83:0/169805128 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8153b50 con 0x7fc5f81607c0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f5ffb700 1 -- 172.21.15.83:0/169805128 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310093001 0 0) 0x7fc5f80d1780 con 0x7fc5f80d0cb0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f67fc700 1 --2- 172.21.15.83:0/169805128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f80d0770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806eab0 tx=0x7fc5ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f73449d93711ec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f5ffb700 1 -- 172.21.15.83:0/169805128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00b870 con 0x7fc5f81607c0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f5ffb700 1 -- 172.21.15.83:0/169805128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec00b9d0 con 0x7fc5f81607c0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.627+0000 7fc5f5ffb700 1 -- 172.21.15.83:0/169805128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00bd60 con 0x7fc5f81607c0 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 -- 172.21.15.83:0/169805128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 msgr2=0x7fc5f80d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/169805128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f80d0770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806eab0 tx=0x7fc5ec009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 -- 172.21.15.83:0/169805128 shutdown_connections 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/169805128 >> v2:172.21.15.90:3300/0 conn(0x7fc5f80ceee0 0x7fc5f8160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/169805128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f80d0770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 -- 172.21.15.83:0/169805128 >> 172.21.15.83:0/169805128 conn(0x7fc5f806de20 msgr2=0x7fc5f806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 -- 172.21.15.83:0/169805128 shutdown_connections 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.628+0000 7fc5fd72a700 1 -- 172.21.15.83:0/169805128 wait complete. 2023-11-28T11:08:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.629+0000 7fc5fd72a700 1 Processor -- start 2023-11-28T11:08:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.629+0000 7fc5fd72a700 1 -- start start 2023-11-28T11:08:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5fd72a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5f80ceee0 0x7fc5f8153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5fd72a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5f80d0cb0 0x7fc5f80c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5fd72a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f8158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5fd72a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f814cd10 con 0x7fc5f81607c0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5fd72a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c9d60 con 0x7fc5f80ceee0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5fd72a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8158aa0 con 0x7fc5f80d0cb0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f8158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f8158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48468/0 (socket says 172.21.15.83:48468) 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 -- 172.21.15.83:0/1497233198 learned_addr learned my addr 172.21.15.83:0/1497233198 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f6ffd700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5f80ceee0 0x7fc5f8153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f67fc700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5f80d0cb0 0x7fc5f80c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 -- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5f80d0cb0 msgr2=0x7fc5f80c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5f80d0cb0 0x7fc5f80c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 -- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5f80ceee0 msgr2=0x7fc5f8153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5f80ceee0 0x7fc5f8153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 -- 172.21.15.83:0/1497233198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec009550 con 0x7fc5f81607c0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f67fc700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5f80d0cb0 0x7fc5f80c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.630+0000 7fc5f77fe700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f8158000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0002230 tx=0x7fc5f000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.631+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000ab70 con 0x7fc5f81607c0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.631+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f000acf0 con 0x7fc5f81607c0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.631+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f806bdb0 con 0x7fc5f81607c0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.631+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0007310 con 0x7fc5f81607c0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.631+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80d14a0 con 0x7fc5f81607c0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.632+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f806c090 con 0x7fc5f81607c0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.632+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5f001a030 con 0x7fc5f81607c0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.632+0000 7fc5df7fe700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc5e40682f0 0x7fc5e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.632+0000 7fc5f6ffd700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc5e40682f0 0x7fc5e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.632+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc5f0085ca0 con 0x7fc5f81607c0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.633+0000 7fc5f6ffd700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc5e40682f0 0x7fc5e406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8006b20 tx=0x7fc5e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.637+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5f008a070 con 0x7fc5f81607c0 2023-11-28T11:08:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.837+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5f8067c20 con 0x7fc5f81607c0 2023-11-28T11:08:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.837+0000 7fc5df7fe700 1 -- 172.21.15.83:0/1497233198 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc5f0018080 con 0x7fc5f81607c0 2023-11-28T11:08:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc5e40682f0 msgr2=0x7fc5e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc5e40682f0 0x7fc5e406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8006b20 tx=0x7fc5e8008040 comp rx=0 tx=0).stop 2023-11-28T11:08:46.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 msgr2=0x7fc5f8158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f8158000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0002230 tx=0x7fc5f000cf50 comp rx=0 tx=0).stop 2023-11-28T11:08:46.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 shutdown_connections 2023-11-28T11:08:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc5e40682f0 0x7fc5e406a7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f81607c0 0x7fc5f8158000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5f80ceee0 0x7fc5f8153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 --2- 172.21.15.83:0/1497233198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5f80d0cb0 0x7fc5f80c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 >> 172.21.15.83:0/1497233198 conn(0x7fc5f806de20 msgr2=0x7fc5f80b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 shutdown_connections 2023-11-28T11:08:46.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:46.839+0000 7fc5fd72a700 1 -- 172.21.15.83:0/1497233198 wait complete. 2023-11-28T11:08:46.843 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:08:46.852 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:08:46.853 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:46.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.030+0000 7fe2148ed700 1 Processor -- start 2023-11-28T11:08:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.030+0000 7fe2148ed700 1 -- start start 2023-11-28T11:08:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe2148ed700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe21015c850 0x7fe2100ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe2148ed700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 0x7fe2100ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe2148ed700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe210161450 con 0x7fe21015cc20 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe2148ed700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2100ce630 con 0x7fe21015c850 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe2148ed700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101608d0 con 0x7fe21006cf90 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe20ed9d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 0x7fe2100ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe20e59c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe21015c850 0x7fe2100ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe20ed9d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 0x7fe2100ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:60912/0 (socket says 172.21.15.83:60912) 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe20ed9d700 1 -- 172.21.15.83:0/1992969535 learned_addr learned my addr 172.21.15.83:0/1992969535 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.031+0000 7fe20dd9b700 1 --1- 172.21.15.83:0/1992969535 >> v1:172.21.15.90:6789/0 conn(0x7fe21015cc20 0x7fe21006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44680/0 (socket says 172.21.15.83:44680) 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20ed9d700 1 -- 172.21.15.83:0/1992969535 >> v2:172.21.15.90:3300/0 conn(0x7fe21015c850 msgr2=0x7fe2100ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20ed9d700 1 --2- 172.21.15.83:0/1992969535 >> v2:172.21.15.90:3300/0 conn(0x7fe21015c850 0x7fe2100ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20ed9d700 1 -- 172.21.15.83:0/1992969535 >> v1:172.21.15.90:6789/0 conn(0x7fe21015cc20 legacy=0x7fe21006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20ed9d700 1 -- 172.21.15.83:0/1992969535 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210160bb0 con 0x7fe21006cf90 2023-11-28T11:08:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20e59c700 1 --2- 172.21.15.83:0/1992969535 >> v2:172.21.15.90:3300/0 conn(0x7fe21015c850 0x7fe2100ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20ed9d700 1 --2- 172.21.15.83:0/1992969535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 0x7fe2100ceff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe208009e50 tx=0x7fe20800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f209cf29fbdc2a8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20d59a700 1 -- 172.21.15.83:0/1992969535 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800e070 con 0x7fe21006cf90 2023-11-28T11:08:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20d59a700 1 -- 172.21.15.83:0/1992969535 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20800b910 con 0x7fe21006cf90 2023-11-28T11:08:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.032+0000 7fe20d59a700 1 -- 172.21.15.83:0/1992969535 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800bcc0 con 0x7fe21006cf90 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 -- 172.21.15.83:0/1992969535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 msgr2=0x7fe2100ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 --2- 172.21.15.83:0/1992969535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 0x7fe2100ceff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe208009e50 tx=0x7fe20800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 -- 172.21.15.83:0/1992969535 shutdown_connections 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 --2- 172.21.15.83:0/1992969535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21006cf90 0x7fe2100ceff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 --2- 172.21.15.83:0/1992969535 >> v2:172.21.15.90:3300/0 conn(0x7fe21015c850 0x7fe2100ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 -- 172.21.15.83:0/1992969535 >> 172.21.15.83:0/1992969535 conn(0x7fe2100cd960 msgr2=0x7fe2100cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 -- 172.21.15.83:0/1992969535 shutdown_connections 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.033+0000 7fe2148ed700 1 -- 172.21.15.83:0/1992969535 wait complete. 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.034+0000 7fe2148ed700 1 Processor -- start 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.034+0000 7fe2148ed700 1 -- start start 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe2148ed700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21006cf90 0x7fe2100b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe2148ed700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21015c850 0x7fe21015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe2148ed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 0x7fe21015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe2148ed700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100c4e70 con 0x7fe21015cc20 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe2148ed700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100c4840 con 0x7fe21006cf90 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe2148ed700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101600e0 con 0x7fe21015c850 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 0x7fe21015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 0x7fe21015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48484/0 (socket says 172.21.15.83:48484) 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 -- 172.21.15.83:0/134233474 learned_addr learned my addr 172.21.15.83:0/134233474 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20dd9b700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21015c850 0x7fe21015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20e59c700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21006cf90 0x7fe2100b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 -- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21015c850 msgr2=0x7fe21015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21015c850 0x7fe21015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 -- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21006cf90 msgr2=0x7fe2100b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21006cf90 0x7fe2100b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.035+0000 7fe20ed9d700 1 -- 172.21.15.83:0/134233474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040094a0 con 0x7fe21015cc20 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.036+0000 7fe20ed9d700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 0x7fe21015e6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe2100723f0 tx=0x7fe208004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.036+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208004800 con 0x7fe21015cc20 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.036+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208004960 con 0x7fe21015cc20 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.036+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe208009b30 con 0x7fe21015cc20 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.036+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20801c680 con 0x7fe21015cc20 2023-11-28T11:08:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.036+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe210161170 con 0x7fe21015cc20 2023-11-28T11:08:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.037+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210160e90 con 0x7fe21015cc20 2023-11-28T11:08:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.037+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe20802c070 con 0x7fe21015cc20 2023-11-28T11:08:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.037+0000 7fe1feffd700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1f00682f0 0x7fe1f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.038+0000 7fe20e59c700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1f00682f0 0x7fe1f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.038+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe208087a20 con 0x7fe21015cc20 2023-11-28T11:08:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.038+0000 7fe20e59c700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1f00682f0 0x7fe1f006a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe204009d70 tx=0x7fe204009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.042+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe208053f10 con 0x7fe21015cc20 2023-11-28T11:08:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.225+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe210160560 con 0x7fe21015cc20 2023-11-28T11:08:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.226+0000 7fe1feffd700 1 -- 172.21.15.83:0/134233474 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fe208057530 con 0x7fe21015cc20 2023-11-28T11:08:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:47.228 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1f00682f0 msgr2=0x7fe1f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1f00682f0 0x7fe1f006a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe204009d70 tx=0x7fe204009210 comp rx=0 tx=0).stop 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 msgr2=0x7fe21015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 0x7fe21015e6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe2100723f0 tx=0x7fe208004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 shutdown_connections 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1f00682f0 0x7fe1f006a7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe21015cc20 0x7fe21015e6e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21006cf90 0x7fe2100b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 --2- 172.21.15.83:0/134233474 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe21015c850 0x7fe21015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 >> 172.21.15.83:0/134233474 conn(0x7fe2100cd960 msgr2=0x7fe21015bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 shutdown_connections 2023-11-28T11:08:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.228+0000 7fe2148ed700 1 -- 172.21.15.83:0/134233474 wait complete. 2023-11-28T11:08:47.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:08:47.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 Processor -- start 2023-11-28T11:08:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 -- start start 2023-11-28T11:08:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 0x7efddc0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7efddc06e370 0x7efddc06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc06d4a0 con 0x7efddc0b4bb0 2023-11-28T11:08:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0b5020 con 0x7efddc06e370 2023-11-28T11:08:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.363+0000 7efde26fe700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0b5ae0 con 0x7efddc15c890 2023-11-28T11:08:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddbfff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 0x7efddc0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddbfff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 0x7efddc0aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:60922/0 (socket says 172.21.15.83:60922) 2023-11-28T11:08:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddbfff700 1 -- 172.21.15.83:0/3933094252 learned_addr learned my addr 172.21.15.83:0/3933094252 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efde0c9b700 1 --2- 172.21.15.83:0/3933094252 >> v2:172.21.15.90:3300/0 conn(0x7efddc06e370 0x7efddc06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efde0c9b700 1 -- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 msgr2=0x7efddc0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efde0c9b700 1 --2- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 0x7efddc0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efde0c9b700 1 -- 172.21.15.83:0/3933094252 >> v1:172.21.15.90:6789/0 conn(0x7efddc0b4bb0 legacy=0x7efddc06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efde0c9b700 1 -- 172.21.15.83:0/3933094252 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc06c400 con 0x7efddc06e370 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddbfff700 1 --2- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 0x7efddc0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efde0c9b700 1 --2- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc06e370 0x7efddc06b150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009d90 tx=0x7efdd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=256198abbf568db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddaffd700 1 -- 172.21.15.83:0/3933094252 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd400e070 con 0x7efddc06e370 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddaffd700 1 -- 172.21.15.83:0/3933094252 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd400bb20 con 0x7efddc06e370 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.364+0000 7efddaffd700 1 -- 172.21.15.83:0/3933094252 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4012430 con 0x7efddc06e370 2023-11-28T11:08:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 -- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc06e370 msgr2=0x7efddc06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 --2- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc06e370 0x7efddc06b150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009d90 tx=0x7efdd400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 -- 172.21.15.83:0/3933094252 shutdown_connections 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 --2- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc06e370 0x7efddc06b150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 --2- 172.21.15.83:0/3933094252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc15c890 0x7efddc0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 -- 172.21.15.83:0/3933094252 >> 172.21.15.83:0/3933094252 conn(0x7efddc0b2b60 msgr2=0x7efddc0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 -- 172.21.15.83:0/3933094252 shutdown_connections 2023-11-28T11:08:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.365+0000 7efde26fe700 1 -- 172.21.15.83:0/3933094252 wait complete. 2023-11-28T11:08:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.366+0000 7efde26fe700 1 Processor -- start 2023-11-28T11:08:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.366+0000 7efde26fe700 1 -- start start 2023-11-28T11:08:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde26fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 0x7efddc14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde26fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc0b4bb0 0x7efddc0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde26fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc15c890 0x7efddc0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde26fe700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc09aab0 con 0x7efddc06e370 2023-11-28T11:08:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde26fe700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc09ad90 con 0x7efddc15c890 2023-11-28T11:08:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde26fe700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc09b070 con 0x7efddc0b4bb0 2023-11-28T11:08:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 0x7efddc14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 0x7efddc14e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48494/0 (socket says 172.21.15.83:48494) 2023-11-28T11:08:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 -- 172.21.15.83:0/3137582354 learned_addr learned my addr 172.21.15.83:0/3137582354 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddb7fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc0b4bb0 0x7efddc0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efde0c9b700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc15c890 0x7efddc0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 -- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc0b4bb0 msgr2=0x7efddc0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc0b4bb0 0x7efddc0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 -- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc15c890 msgr2=0x7efddc0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc15c890 0x7efddc0cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.367+0000 7efddbfff700 1 -- 172.21.15.83:0/3137582354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc009500 con 0x7efddc06e370 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.368+0000 7efddbfff700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 0x7efddc14e220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00e9a0 tx=0x7efdcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.368+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc005a20 con 0x7efddc06e370 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.368+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc005b80 con 0x7efddc06e370 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.368+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd4009a70 con 0x7efddc06e370 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.368+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00c690 con 0x7efddc06e370 2023-11-28T11:08:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.368+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc0b5520 con 0x7efddc06e370 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.369+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc0b5240 con 0x7efddc06e370 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.369+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efdcc022070 con 0x7efddc06e370 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.369+0000 7efdd8ff9700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdbc0682f0 0x7efdbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.369+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efdcc085fb0 con 0x7efddc06e370 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.370+0000 7efddb7fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdbc0682f0 0x7efdbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.370+0000 7efddb7fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdbc0682f0 0x7efdbc06a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efddc09c650 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.374+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efdcc052520 con 0x7efddc06e370 2023-11-28T11:08:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.553+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efddc09b9a0 con 0x7efddc06e370 2023-11-28T11:08:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.553+0000 7efdd8ff9700 1 -- 172.21.15.83:0/3137582354 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7efdcc020020 con 0x7efddc06e370 2023-11-28T11:08:47.555 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdbc0682f0 msgr2=0x7efdbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdbc0682f0 0x7efdbc06a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efddc09c650 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 msgr2=0x7efddc14e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 0x7efddc14e220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00e9a0 tx=0x7efdcc00ecb0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 shutdown_connections 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efddc15c890 0x7efddc0cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdbc0682f0 0x7efdbc06a7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efddc06e370 0x7efddc14e220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 --2- 172.21.15.83:0/3137582354 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efddc0b4bb0 0x7efddc0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 >> 172.21.15.83:0/3137582354 conn(0x7efddc0b2b60 msgr2=0x7efddc0c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 shutdown_connections 2023-11-28T11:08:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.555+0000 7efde26fe700 1 -- 172.21.15.83:0/3137582354 wait complete. 2023-11-28T11:08:47.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.690+0000 7ff377433700 1 Processor -- start 2023-11-28T11:08:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.690+0000 7ff377433700 1 -- start start 2023-11-28T11:08:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff377433700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015e5c0 0x7ff37015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff377433700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff37015b3b0 0x7ff3700d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff377433700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff37015b820 0x7ff3700d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff377433700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff370151a30 con 0x7ff37015b3b0 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff377433700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3700cb460 con 0x7ff37015b820 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff377433700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700cb740 con 0x7ff37015e5c0 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff3749ce700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff37015b3b0 0x7ff3700d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff3749ce700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff37015b3b0 0x7ff3700d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48502/0 (socket says 172.21.15.83:48502) 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff3749ce700 1 -- 172.21.15.83:0/3237358607 learned_addr learned my addr 172.21.15.83:0/3237358607 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff3751cf700 1 --2- 172.21.15.83:0/3237358607 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015e5c0 0x7ff37015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.691+0000 7ff3759d0700 1 --2- 172.21.15.83:0/3237358607 >> v2:172.21.15.90:3300/0 conn(0x7ff37015b820 0x7ff3700d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3749ce700 1 -- 172.21.15.83:0/3237358607 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015e5c0 msgr2=0x7ff37015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3749ce700 1 --2- 172.21.15.83:0/3237358607 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015e5c0 0x7ff37015e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3749ce700 1 -- 172.21.15.83:0/3237358607 >> v2:172.21.15.90:3300/0 conn(0x7ff37015b820 msgr2=0x7ff3700d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3749ce700 1 --2- 172.21.15.83:0/3237358607 >> v2:172.21.15.90:3300/0 conn(0x7ff37015b820 0x7ff3700d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3749ce700 1 -- 172.21.15.83:0/3237358607 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37015abb0 con 0x7ff37015b3b0 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3759d0700 1 --2- 172.21.15.83:0/3237358607 >> v2:172.21.15.90:3300/0 conn(0x7ff37015b820 0x7ff3700d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff3749ce700 1 --2- 172.21.15.83:0/3237358607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37015b3b0 0x7ff3700d0ae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff37006f670 tx=0x7ff36c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bef3f3a80b4d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff367fff700 1 -- 172.21.15.83:0/3237358607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00b870 con 0x7ff37015b3b0 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff367fff700 1 -- 172.21.15.83:0/3237358607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c00b9d0 con 0x7ff37015b3b0 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff367fff700 1 -- 172.21.15.83:0/3237358607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00bd60 con 0x7ff37015b3b0 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 -- 172.21.15.83:0/3237358607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37015b3b0 msgr2=0x7ff3700d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 --2- 172.21.15.83:0/3237358607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37015b3b0 0x7ff3700d0ae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff37006f670 tx=0x7ff36c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 -- 172.21.15.83:0/3237358607 shutdown_connections 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 --2- 172.21.15.83:0/3237358607 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015e5c0 0x7ff37015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 --2- 172.21.15.83:0/3237358607 >> v2:172.21.15.90:3300/0 conn(0x7ff37015b820 0x7ff3700d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 --2- 172.21.15.83:0/3237358607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37015b3b0 0x7ff3700d0ae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 -- 172.21.15.83:0/3237358607 >> 172.21.15.83:0/3237358607 conn(0x7ff37006ea50 msgr2=0x7ff370070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.692+0000 7ff377433700 1 -- 172.21.15.83:0/3237358607 shutdown_connections 2023-11-28T11:08:47.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.693+0000 7ff377433700 1 -- 172.21.15.83:0/3237358607 wait complete. 2023-11-28T11:08:47.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 Processor -- start 2023-11-28T11:08:47.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 -- start start 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015b820 0x7ff370151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37015e5c0 0x7ff370157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 0x7ff37014c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff370155fd0 con 0x7ff370150280 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701562b0 con 0x7ff37015e5c0 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff377433700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff370156590 con 0x7ff37015b820 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.694+0000 7ff3759d0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 0x7ff37014c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 0x7ff37014c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48508/0 (socket says 172.21.15.83:48508) 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 -- 172.21.15.83:0/992705591 learned_addr learned my addr 172.21.15.83:0/992705591 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3751cf700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015b820 0x7ff370151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3749ce700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37015e5c0 0x7ff370157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 -- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015b820 msgr2=0x7ff370151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015b820 0x7ff370151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 -- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37015e5c0 msgr2=0x7ff370157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37015e5c0 0x7ff370157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 -- 172.21.15.83:0/992705591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c009550 con 0x7ff370150280 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3751cf700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015b820 0x7ff370151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3749ce700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37015e5c0 0x7ff370157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff3759d0700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 0x7ff37014c450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff36800cdf0 tx=0x7ff36800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800ada0 con 0x7ff370150280 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff368003a20 con 0x7ff370150280 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37015eed0 con 0x7ff370150280 2023-11-28T11:08:47.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368003dd0 con 0x7ff370150280 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.695+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37015ae90 con 0x7ff370150280 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.696+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3700cba20 con 0x7ff370150280 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.697+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff368003f30 con 0x7ff370150280 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.697+0000 7ff365ffb700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff3540682f0 0x7ff35406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.697+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff368086450 con 0x7ff370150280 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.697+0000 7ff3751cf700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff3540682f0 0x7ff35406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.698+0000 7ff3751cf700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff3540682f0 0x7ff35406a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff360009a00 tx=0x7ff360008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.701+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff36804d5e0 con 0x7ff370150280 2023-11-28T11:08:47.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.885+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3700742b0 con 0x7ff370150280 2023-11-28T11:08:47.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.886+0000 7ff365ffb700 1 -- 172.21.15.83:0/992705591 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff368052960 con 0x7ff370150280 2023-11-28T11:08:47.887 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:08:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff3540682f0 msgr2=0x7ff35406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff3540682f0 0x7ff35406a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff360009a00 tx=0x7ff360008040 comp rx=0 tx=0).stop 2023-11-28T11:08:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 msgr2=0x7ff37014c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 0x7ff37014c450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff36800cdf0 tx=0x7ff36800ced0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 shutdown_connections 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff3540682f0 0x7ff35406a7a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff370150280 0x7ff37014c450 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff37015b820 0x7ff370151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 --2- 172.21.15.83:0/992705591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37015e5c0 0x7ff370157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 >> 172.21.15.83:0/992705591 conn(0x7ff37015b3b0 msgr2=0x7ff37015c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 shutdown_connections 2023-11-28T11:08:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:47.888+0000 7ff377433700 1 -- 172.21.15.83:0/992705591 wait complete. 2023-11-28T11:08:47.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.022+0000 7f5fb0ddb700 1 Processor -- start 2023-11-28T11:08:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.022+0000 7f5fb0ddb700 1 -- start start 2023-11-28T11:08:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5fb0ddb700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5fac0aefd0 0x7f5fac0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5fb0ddb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5fac0ccd30 0x7f5fac0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5fb0ddb700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac0b4bb0 con 0x7f5fac0cd1c0 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5fb0ddb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac0d0af0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5fb0ddb700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac15ac30 con 0x7f5fac0aefd0 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5faad9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5fac0ccd30 0x7f5fac0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5faad9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5fac0ccd30 0x7f5fac0cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48518/0 (socket says 172.21.15.83:48518) 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.023+0000 7f5faad9d700 1 -- 172.21.15.83:0/3716672724 learned_addr learned my addr 172.21.15.83:0/3716672724 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faa59c700 1 --2- 172.21.15.83:0/3716672724 >> v2:172.21.15.90:3300/0 conn(0x7f5fac0aefd0 0x7f5fac0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faad9d700 1 -- 172.21.15.83:0/3716672724 >> v2:172.21.15.90:3300/0 conn(0x7f5fac0aefd0 msgr2=0x7f5fac0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faad9d700 1 --2- 172.21.15.83:0/3716672724 >> v2:172.21.15.90:3300/0 conn(0x7f5fac0aefd0 0x7f5fac0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faad9d700 1 -- 172.21.15.83:0/3716672724 >> v1:172.21.15.90:6789/0 conn(0x7f5fac0cd1c0 legacy=0x7f5fac0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faad9d700 1 -- 172.21.15.83:0/3716672724 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0b4e00 con 0x7f5fac0ccd30 2023-11-28T11:08:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faa59c700 1 --2- 172.21.15.83:0/3716672724 >> v2:172.21.15.90:3300/0 conn(0x7f5fac0aefd0 0x7f5fac0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5faad9d700 1 --2- 172.21.15.83:0/3716672724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac0cd990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0cfb70 tx=0x7f5f9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9176d2a7da273d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5fa959a700 1 -- 172.21.15.83:0/3716672724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c004230 con 0x7f5fac0ccd30 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5fa959a700 1 -- 172.21.15.83:0/3716672724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c004390 con 0x7f5fac0ccd30 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.024+0000 7f5fa959a700 1 -- 172.21.15.83:0/3716672724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0036a0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3716672724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 msgr2=0x7f5fac0cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3716672724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac0cd990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0cfb70 tx=0x7f5f9c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3716672724 shutdown_connections 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3716672724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac0cd990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3716672724 >> v2:172.21.15.90:3300/0 conn(0x7f5fac0aefd0 0x7f5fac0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3716672724 >> 172.21.15.83:0/3716672724 conn(0x7f5fac0b2b60 msgr2=0x7f5fac0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3716672724 shutdown_connections 2023-11-28T11:08:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.025+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3716672724 wait complete. 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.026+0000 7f5fb0ddb700 1 Processor -- start 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.026+0000 7f5fb0ddb700 1 -- start start 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fb0ddb700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5fac0aefd0 0x7f5fac0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fb0ddb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fb0ddb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fac0cd1c0 0x7f5fac1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fb0ddb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0b03f0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fb0ddb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0b0640 con 0x7f5fac0cd1c0 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fb0ddb700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0b0920 con 0x7f5fac0aefd0 2023-11-28T11:08:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.027+0000 7f5fa9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac15bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48522/0 (socket says 172.21.15.83:48522) 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 -- 172.21.15.83:0/3586731654 learned_addr learned my addr 172.21.15.83:0/3586731654 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5faa59c700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5fac0aefd0 0x7f5fac0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5faad9d700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fac0cd1c0 0x7f5fac1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 -- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5fac0aefd0 msgr2=0x7f5fac0c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5fac0aefd0 0x7f5fac0c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 -- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fac0cd1c0 msgr2=0x7f5fac1f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fac0cd1c0 0x7f5fac1f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 -- 172.21.15.83:0/3586731654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c009550 con 0x7f5fac0ccd30 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fa9d9b700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac15bb40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000b450 tx=0x7f5fa000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0011830 con 0x7f5fac0ccd30 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0011990 con 0x7f5fac0ccd30 2023-11-28T11:08:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac0b53c0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0011d40 con 0x7f5fac0ccd30 2023-11-28T11:08:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.028+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac0b50e0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.029+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac0b56a0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.030+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fa000f050 con 0x7f5fac0ccd30 2023-11-28T11:08:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.030+0000 7f5f9affd700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.030+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5fa0085a10 con 0x7f5fac0ccd30 2023-11-28T11:08:48.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.030+0000 7f5faa59c700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.031+0000 7f5faa59c700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400b5f0 tx=0x7f5fa400e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.034+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5fa0051f80 con 0x7f5fac0ccd30 2023-11-28T11:08:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.219+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fac0b37b0 con 0x7f5fac0ccd30 2023-11-28T11:08:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.219+0000 7f5f9affd700 1 -- 172.21.15.83:0/3586731654 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fa000be60 con 0x7f5fac0ccd30 2023-11-28T11:08:48.221 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:08:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f8c0682f0 msgr2=0x7f5f8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400b5f0 tx=0x7f5fa400e040 comp rx=0 tx=0).stop 2023-11-28T11:08:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 msgr2=0x7f5fac15bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac15bb40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000b450 tx=0x7f5fa000b760 comp rx=0 tx=0).stop 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 shutdown_connections 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fac0cd1c0 0x7f5fac1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5fac0aefd0 0x7f5fac0c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 --2- 172.21.15.83:0/3586731654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0ccd30 0x7f5fac15bb40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 >> 172.21.15.83:0/3586731654 conn(0x7f5fac0b2b60 msgr2=0x7f5fac15a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 shutdown_connections 2023-11-28T11:08:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.222+0000 7f5fb0ddb700 1 -- 172.21.15.83:0/3586731654 wait complete. 2023-11-28T11:08:48.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.356+0000 7f5f99dab700 1 Processor -- start 2023-11-28T11:08:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.356+0000 7f5f99dab700 1 -- start start 2023-11-28T11:08:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f99dab700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5f940aee20 0x7f5f940b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f99dab700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f9406c370 0x7f5f9406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f99dab700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f940b5b00 con 0x7f5f940af1f0 2023-11-28T11:08:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f99dab700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9406c800 con 0x7f5f940aee20 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f99dab700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9406cae0 con 0x7f5f9406c370 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f93fff700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f5f940af1f0 0x7f5f940cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44738/0 (socket says 172.21.15.83:44738) 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f93fff700 1 -- 172.21.15.83:0/4246246483 learned_addr learned my addr 172.21.15.83:0/4246246483 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f92ffd700 1 --2- 172.21.15.83:0/4246246483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f9406c370 0x7f5f9406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.357+0000 7f5f937fe700 1 --2- 172.21.15.83:0/4246246483 >> v2:172.21.15.90:3300/0 conn(0x7f5f940aee20 0x7f5f940b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f927fc700 1 -- 172.21.15.83:0/4246246483 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110008403 0 0) 0x7f5f8c001490 con 0x7f5f940af1f0 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f927fc700 1 -- 172.21.15.83:0/4246246483 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f7c0035b0 con 0x7f5f940af1f0 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f92ffd700 1 -- 172.21.15.83:0/4246246483 >> v2:172.21.15.90:3300/0 conn(0x7f5f940aee20 msgr2=0x7f5f940b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f92ffd700 1 --2- 172.21.15.83:0/4246246483 >> v2:172.21.15.90:3300/0 conn(0x7f5f940aee20 0x7f5f940b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f92ffd700 1 -- 172.21.15.83:0/4246246483 >> v1:172.21.15.90:6789/0 conn(0x7f5f940af1f0 legacy=0x7f5f940cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f92ffd700 1 -- 172.21.15.83:0/4246246483 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9406cdc0 con 0x7f5f9406c370 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f92ffd700 1 --2- 172.21.15.83:0/4246246483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f9406c370 0x7f5f9406ed80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009840 tx=0x7f5f84009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4119e3b2110130d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f927fc700 1 -- 172.21.15.83:0/4246246483 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84011830 con 0x7f5f9406c370 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f927fc700 1 -- 172.21.15.83:0/4246246483 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84011990 con 0x7f5f9406c370 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.358+0000 7f5f927fc700 1 -- 172.21.15.83:0/4246246483 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84011d20 con 0x7f5f9406c370 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 -- 172.21.15.83:0/4246246483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f9406c370 msgr2=0x7f5f9406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 --2- 172.21.15.83:0/4246246483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f9406c370 0x7f5f9406ed80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009840 tx=0x7f5f84009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 -- 172.21.15.83:0/4246246483 shutdown_connections 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 --2- 172.21.15.83:0/4246246483 >> v2:172.21.15.90:3300/0 conn(0x7f5f940aee20 0x7f5f940b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 --2- 172.21.15.83:0/4246246483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f9406c370 0x7f5f9406ed80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 -- 172.21.15.83:0/4246246483 >> 172.21.15.83:0/4246246483 conn(0x7f5f940b2b60 msgr2=0x7f5f940b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 -- 172.21.15.83:0/4246246483 shutdown_connections 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.359+0000 7f5f99dab700 1 -- 172.21.15.83:0/4246246483 wait complete. 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.360+0000 7f5f99dab700 1 Processor -- start 2023-11-28T11:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.360+0000 7f5f99dab700 1 -- start start 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f99dab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 0x7f5f94145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f99dab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f940aee20 0x7f5f941f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f99dab700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f940af1f0 0x7f5f94143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f99dab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9415ad90 con 0x7f5f9406c370 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f99dab700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941464a0 con 0x7f5f940aee20 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f99dab700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94146780 con 0x7f5f940af1f0 2023-11-28T11:08:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 0x7f5f94145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f940aee20 0x7f5f941f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 0x7f5f94145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48534/0 (socket says 172.21.15.83:48534) 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 -- 172.21.15.83:0/3473564306 learned_addr learned my addr 172.21.15.83:0/3473564306 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f93fff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f940af1f0 0x7f5f94143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 -- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f940af1f0 msgr2=0x7f5f94143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f940af1f0 0x7f5f94143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 -- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f940aee20 msgr2=0x7f5f941f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f940aee20 0x7f5f941f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 -- 172.21.15.83:0/3473564306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f840094a0 con 0x7f5f9406c370 2023-11-28T11:08:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f92ffd700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f940aee20 0x7f5f941f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f937fe700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 0x7f5f94145e80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e990 tx=0x7f5f8800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.361+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f880096f0 con 0x7f5f9406c370 2023-11-28T11:08:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.362+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8800c690 con 0x7f5f9406c370 2023-11-28T11:08:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.362+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9406fc70 con 0x7f5f9406c370 2023-11-28T11:08:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.362+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800ca00 con 0x7f5f9406c370 2023-11-28T11:08:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.362+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9406d0a0 con 0x7f5f9406c370 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.362+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9406d380 con 0x7f5f9406c370 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.363+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f88023070 con 0x7f5f9406c370 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.364+0000 7f5f7bfff700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f800682f0 0x7f5f8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.364+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5f88086600 con 0x7f5f9406c370 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.364+0000 7f5f92ffd700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f800682f0 0x7f5f8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.364+0000 7f5f92ffd700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f800682f0 0x7f5f8006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f84000c00 tx=0x7f5f84008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.367+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f88052b70 con 0x7f5f9406c370 2023-11-28T11:08:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.550+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f9415b070 con 0x7f5f9406c370 2023-11-28T11:08:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.550+0000 7f5f7bfff700 1 -- 172.21.15.83:0/3473564306 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f88056190 con 0x7f5f9406c370 2023-11-28T11:08:48.552 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:08:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.552+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f800682f0 msgr2=0x7f5f8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.552+0000 7f5f99dab700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f800682f0 0x7f5f8006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f84000c00 tx=0x7f5f84008030 comp rx=0 tx=0).stop 2023-11-28T11:08:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.552+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 msgr2=0x7f5f94145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.552+0000 7f5f99dab700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 0x7f5f94145e80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e990 tx=0x7f5f8800eca0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 shutdown_connections 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f940af1f0 0x7f5f94143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5f800682f0 0x7f5f8006a7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9406c370 0x7f5f94145e80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 --2- 172.21.15.83:0/3473564306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f940aee20 0x7f5f941f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 >> 172.21.15.83:0/3473564306 conn(0x7f5f940b2b60 msgr2=0x7f5f940afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 shutdown_connections 2023-11-28T11:08:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.553+0000 7f5f99dab700 1 -- 172.21.15.83:0/3473564306 wait complete. 2023-11-28T11:08:48.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:08:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.687+0000 7fa53d9e4700 1 Processor -- start 2023-11-28T11:08:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.687+0000 7fa53d9e4700 1 -- start start 2023-11-28T11:08:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa53d9e4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5380d2880 0x7fa53806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa53d9e4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5380cd520 con 0x7fa5380b8570 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa53d9e4700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5380cf400 con 0x7fa5380b4940 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa53d9e4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5380cd7c0 con 0x7fa5380d2880 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5380d2880 0x7fa53806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5380d2880 0x7fa53806a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48544/0 (socket says 172.21.15.83:48544) 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 -- 172.21.15.83:0/46781584 learned_addr learned my addr 172.21.15.83:0/46781584 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa536ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa5380b8570 0x7fa538160c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46402/0 (socket says 172.21.15.83:46402) 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 -- 172.21.15.83:0/46781584 >> v1:172.21.15.90:6789/0 conn(0x7fa5380b4940 legacy=0x7fa5380d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 -- 172.21.15.83:0/46781584 >> v1:172.21.15.83:6789/0 conn(0x7fa5380b8570 legacy=0x7fa538160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 -- 172.21.15.83:0/46781584 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380cdaa0 con 0x7fa5380d2880 2023-11-28T11:08:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa535ffb700 1 -- 172.21.15.83:0/46781584 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564704149 0 0) 0x7fa5380cd520 con 0x7fa5380b8570 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.688+0000 7fa5367fc700 1 --2- 172.21.15.83:0/46781584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380d2880 0x7fa53806a270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098f0 tx=0x7fa528009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68087e326b3ccb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa535ffb700 1 -- 172.21.15.83:0/46781584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52800b8a0 con 0x7fa5380d2880 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa535ffb700 1 -- 172.21.15.83:0/46781584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52800ba00 con 0x7fa5380d2880 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa535ffb700 1 -- 172.21.15.83:0/46781584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52800bd90 con 0x7fa5380d2880 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 -- 172.21.15.83:0/46781584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380d2880 msgr2=0x7fa53806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/46781584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380d2880 0x7fa53806a270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098f0 tx=0x7fa528009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 -- 172.21.15.83:0/46781584 shutdown_connections 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/46781584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380d2880 0x7fa53806a270 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 -- 172.21.15.83:0/46781584 >> 172.21.15.83:0/46781584 conn(0x7fa5380d30a0 msgr2=0x7fa5380d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 -- 172.21.15.83:0/46781584 shutdown_connections 2023-11-28T11:08:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.689+0000 7fa53d9e4700 1 -- 172.21.15.83:0/46781584 wait complete. 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.690+0000 7fa53d9e4700 1 Processor -- start 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 -- start start 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5380b4940 0x7fa53815b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 0x7fa53815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5380d2880 0x7fa53814b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53814ca10 con 0x7fa5380b8570 2023-11-28T11:08:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53815c990 con 0x7fa5380d2880 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa53d9e4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53814c450 con 0x7fa5380b4940 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa5367fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 0x7fa53815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa5367fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 0x7fa53815b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48550/0 (socket says 172.21.15.83:48550) 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.691+0000 7fa5367fc700 1 -- 172.21.15.83:0/60655309 learned_addr learned my addr 172.21.15.83:0/60655309 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa536ffd700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5380b4940 0x7fa53815b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5377fe700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5380d2880 0x7fa53814b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5367fc700 1 -- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5380b4940 msgr2=0x7fa53815b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5367fc700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5380b4940 0x7fa53815b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5367fc700 1 -- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5380d2880 msgr2=0x7fa53814b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5367fc700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5380d2880 0x7fa53814b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5367fc700 1 -- 172.21.15.83:0/60655309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528009550 con 0x7fa5380b8570 2023-11-28T11:08:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa536ffd700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5380b4940 0x7fa53815b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5377fe700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5380d2880 0x7fa53814b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa5367fc700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 0x7fa53815b5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa528014040 tx=0x7fa528004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5280038a0 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52801e070 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5380cf0f0 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528003c20 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.692+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5380cf6e0 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.693+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5380b8940 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.693+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa52800f050 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.694+0000 7fa51ffff700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5240682f0 0x7fa52406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.694+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa528020070 con 0x7fa5380b8570 2023-11-28T11:08:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.694+0000 7fa536ffd700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5240682f0 0x7fa52406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.694+0000 7fa536ffd700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5240682f0 0x7fa52406a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000f60 tx=0x7fa52c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.698+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa528053af0 con 0x7fa5380b8570 2023-11-28T11:08:48.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.881+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa53815cad0 con 0x7fa5380b8570 2023-11-28T11:08:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.881+0000 7fa51ffff700 1 -- 172.21.15.83:0/60655309 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa53815cad0 con 0x7fa5380b8570 2023-11-28T11:08:48.883 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:08:48.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5240682f0 msgr2=0x7fa52406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5240682f0 0x7fa52406a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000f60 tx=0x7fa52c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 msgr2=0x7fa53815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 0x7fa53815b5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa528014040 tx=0x7fa528004750 comp rx=0 tx=0).stop 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 shutdown_connections 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5380d2880 0x7fa53814b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5380b4940 0x7fa53815b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5240682f0 0x7fa52406a7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 --2- 172.21.15.83:0/60655309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5380b8570 0x7fa53815b5b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.883+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 >> 172.21.15.83:0/60655309 conn(0x7fa5380d30a0 msgr2=0x7fa5380b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.884+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 shutdown_connections 2023-11-28T11:08:48.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:48.884+0000 7fa53d9e4700 1 -- 172.21.15.83:0/60655309 wait complete. 2023-11-28T11:08:48.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:08:49.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.016+0000 7f03226ac700 1 Processor -- start 2023-11-28T11:08:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.016+0000 7f03226ac700 1 -- start start 2023-11-28T11:08:49.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f03226ac700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f031c15c890 0x7f031c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f03226ac700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f031c0b4bb0 0x7f031c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f03226ac700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f03226ac700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c06e620 con 0x7f031c0b4bb0 2023-11-28T11:08:49.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f03226ac700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c0b5020 con 0x7f031c15c890 2023-11-28T11:08:49.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f03226ac700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c06d4a0 con 0x7f031c06e1a0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f031c0b4bb0 0x7f031c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f031c0b4bb0 0x7f031c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48558/0 (socket says 172.21.15.83:48558) 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 -- 172.21.15.83:0/2113624493 learned_addr learned my addr 172.21.15.83:0/2113624493 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031bfff700 1 --2- 172.21.15.83:0/2113624493 >> v2:172.21.15.90:3300/0 conn(0x7f031c15c890 0x7f031c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f0320c49700 1 --2- 172.21.15.83:0/2113624493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 -- 172.21.15.83:0/2113624493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 msgr2=0x7f031c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 --2- 172.21.15.83:0/2113624493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 -- 172.21.15.83:0/2113624493 >> v2:172.21.15.90:3300/0 conn(0x7f031c15c890 msgr2=0x7f031c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 --2- 172.21.15.83:0/2113624493 >> v2:172.21.15.90:3300/0 conn(0x7f031c15c890 0x7f031c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.017+0000 7f031b7fe700 1 -- 172.21.15.83:0/2113624493 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c0b5ae0 con 0x7f031c0b4bb0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f031b7fe700 1 --2- 172.21.15.83:0/2113624493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c06dc60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f031c0715b0 tx=0x7f030c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dd9f64bca2d254 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f031affd700 1 -- 172.21.15.83:0/2113624493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00b870 con 0x7f031c0b4bb0 2023-11-28T11:08:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f031affd700 1 -- 172.21.15.83:0/2113624493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00b9d0 con 0x7f031c0b4bb0 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f031affd700 1 -- 172.21.15.83:0/2113624493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00bd60 con 0x7f031c0b4bb0 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 -- 172.21.15.83:0/2113624493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 msgr2=0x7f031c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 --2- 172.21.15.83:0/2113624493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c06dc60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f031c0715b0 tx=0x7f030c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 -- 172.21.15.83:0/2113624493 shutdown_connections 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 --2- 172.21.15.83:0/2113624493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 --2- 172.21.15.83:0/2113624493 >> v2:172.21.15.90:3300/0 conn(0x7f031c15c890 0x7f031c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 --2- 172.21.15.83:0/2113624493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c06dc60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 -- 172.21.15.83:0/2113624493 >> 172.21.15.83:0/2113624493 conn(0x7f031c0b2b60 msgr2=0x7f031c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 -- 172.21.15.83:0/2113624493 shutdown_connections 2023-11-28T11:08:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.018+0000 7f03226ac700 1 -- 172.21.15.83:0/2113624493 wait complete. 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.019+0000 7f03226ac700 1 Processor -- start 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 -- start start 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031c15c890 0x7f031c0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0ccdf0 con 0x7f031c0b4bb0 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0c65a0 con 0x7f031c15c890 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f03226ac700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0c62c0 con 0x7f031c06e1a0 2023-11-28T11:08:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c0c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48564/0 (socket says 172.21.15.83:48564) 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.020+0000 7f031b7fe700 1 -- 172.21.15.83:0/4006567506 learned_addr learned my addr 172.21.15.83:0/4006567506 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f0320c49700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031c15c890 0x7f031c0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031bfff700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031b7fe700 1 -- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 msgr2=0x7f031c1f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031b7fe700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031b7fe700 1 -- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031c15c890 msgr2=0x7f031c0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031b7fe700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031c15c890 0x7f031c0c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031b7fe700 1 -- 172.21.15.83:0/4006567506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009550 con 0x7f031c0b4bb0 2023-11-28T11:08:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f0320c49700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031c15c890 0x7f031c0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031bfff700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f031b7fe700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c0c6ce0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f030c014040 tx=0x7f030c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c004520 con 0x7f031c0b4bb0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c01e070 con 0x7f031c0b4bb0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c0b5240 con 0x7f031c0b4bb0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0038c0 con 0x7f031c0b4bb0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.021+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c06c400 con 0x7f031c0b4bb0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.022+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031c0b5520 con 0x7f031c0b4bb0 2023-11-28T11:08:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.022+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f030c00f050 con 0x7f031c0b4bb0 2023-11-28T11:08:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.023+0000 7f0318ff9700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03080682f0 0x7f030806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.023+0000 7f031bfff700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03080682f0 0x7f030806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.023+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f030c020070 con 0x7f031c0b4bb0 2023-11-28T11:08:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.023+0000 7f031bfff700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03080682f0 0x7f030806a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f031c1f5b80 tx=0x7f0310008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.027+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f030c053a90 con 0x7f031c0b4bb0 2023-11-28T11:08:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.224+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f031c0ae960 con 0x7f031c0b4bb0 2023-11-28T11:08:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.224+0000 7f0318ff9700 1 -- 172.21.15.83:0/4006567506 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f030c0243e0 con 0x7f031c0b4bb0 2023-11-28T11:08:49.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03080682f0 msgr2=0x7f030806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03080682f0 0x7f030806a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f031c1f5b80 tx=0x7f0310008040 comp rx=0 tx=0).stop 2023-11-28T11:08:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 msgr2=0x7f031c0c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c0c6ce0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f030c014040 tx=0x7f030c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 shutdown_connections 2023-11-28T11:08:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031c15c890 0x7f031c0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031c06e1a0 0x7f031c1f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03080682f0 0x7f030806a7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 --2- 172.21.15.83:0/4006567506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f031c0b4bb0 0x7f031c0c6ce0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 >> 172.21.15.83:0/4006567506 conn(0x7f031c0b2b60 msgr2=0x7f031c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.226+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 shutdown_connections 2023-11-28T11:08:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.227+0000 7f03226ac700 1 -- 172.21.15.83:0/4006567506 wait complete. 2023-11-28T11:08:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:08:49.240 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2023-11-28T11:08:49.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:49.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:49.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[110958]: fuse finished with error 0 and tester_r 0 2023-11-28T11:08:49.279 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:08:49.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:08:49.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:49.367 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-28T11:08:49.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.549+0000 7f0a52a54700 1 Processor -- start 2023-11-28T11:08:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.549+0000 7f0a52a54700 1 -- start start 2023-11-28T11:08:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a52a54700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0a4c06a950 0x7f0a4c15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a52a54700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a4c0b4e40 0x7f0a4c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a52a54700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c0ca070 0x7f0a4c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a52a54700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c15b160 con 0x7f0a4c0b4e40 2023-11-28T11:08:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a52a54700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c15b440 con 0x7f0a4c06a950 2023-11-28T11:08:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a52a54700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c15b720 con 0x7f0a4c0ca070 2023-11-28T11:08:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a4b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a4c0b4e40 0x7f0a4c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a4b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a4c0b4e40 0x7f0a4c0c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48572/0 (socket says 172.21.15.83:48572) 2023-11-28T11:08:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.550+0000 7f0a4b7fe700 1 -- 172.21.15.83:0/288569902 learned_addr learned my addr 172.21.15.83:0/288569902 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4bfff700 1 --2- 172.21.15.83:0/288569902 >> v2:172.21.15.90:3300/0 conn(0x7f0a4c06a950 0x7f0a4c15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a50ff1700 1 --2- 172.21.15.83:0/288569902 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c0ca070 0x7f0a4c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4b7fe700 1 -- 172.21.15.83:0/288569902 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c0ca070 msgr2=0x7f0a4c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4b7fe700 1 --2- 172.21.15.83:0/288569902 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c0ca070 0x7f0a4c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4b7fe700 1 -- 172.21.15.83:0/288569902 >> v2:172.21.15.90:3300/0 conn(0x7f0a4c06a950 msgr2=0x7f0a4c15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4b7fe700 1 --2- 172.21.15.83:0/288569902 >> v2:172.21.15.90:3300/0 conn(0x7f0a4c06a950 0x7f0a4c15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4b7fe700 1 -- 172.21.15.83:0/288569902 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c071450 con 0x7f0a4c0b4e40 2023-11-28T11:08:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4b7fe700 1 --2- 172.21.15.83:0/288569902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c0c9b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c06e800 tx=0x7f0a3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee88507cb4bea72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4affd700 1 -- 172.21.15.83:0/288569902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00b870 con 0x7f0a4c0b4e40 2023-11-28T11:08:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4affd700 1 -- 172.21.15.83:0/288569902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c00b9d0 con 0x7f0a4c0b4e40 2023-11-28T11:08:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a4affd700 1 -- 172.21.15.83:0/288569902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00bd60 con 0x7f0a4c0b4e40 2023-11-28T11:08:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a52a54700 1 -- 172.21.15.83:0/288569902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0b4e40 msgr2=0x7f0a4c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.551+0000 7f0a52a54700 1 --2- 172.21.15.83:0/288569902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c0c9b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c06e800 tx=0x7f0a3c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 -- 172.21.15.83:0/288569902 shutdown_connections 2023-11-28T11:08:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 --2- 172.21.15.83:0/288569902 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c0ca070 0x7f0a4c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 --2- 172.21.15.83:0/288569902 >> v2:172.21.15.90:3300/0 conn(0x7f0a4c06a950 0x7f0a4c15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 --2- 172.21.15.83:0/288569902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c0c9b30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 -- 172.21.15.83:0/288569902 >> 172.21.15.83:0/288569902 conn(0x7f0a4c0cf4e0 msgr2=0x7f0a4c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 -- 172.21.15.83:0/288569902 shutdown_connections 2023-11-28T11:08:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.552+0000 7f0a52a54700 1 -- 172.21.15.83:0/288569902 wait complete. 2023-11-28T11:08:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.553+0000 7f0a52a54700 1 Processor -- start 2023-11-28T11:08:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.553+0000 7f0a52a54700 1 -- start start 2023-11-28T11:08:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c06a950 0x7f0a4c1ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 0x7f0a4c1e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c1efe00 con 0x7f0a4c0ca070 2023-11-28T11:08:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c15d0e0 con 0x7f0a4c0b4e40 2023-11-28T11:08:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c1e6af0 con 0x7f0a4c06a950 2023-11-28T11:08:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 0x7f0a4c1e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 0x7f0a4c1e62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48580/0 (socket says 172.21.15.83:48580) 2023-11-28T11:08:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 -- 172.21.15.83:0/4243537778 learned_addr learned my addr 172.21.15.83:0/4243537778 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a4b7fe700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a4bfff700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c06a950 0x7f0a4c1ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 -- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c06a950 msgr2=0x7f0a4c1ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c06a950 0x7f0a4c1ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 -- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a4c0b4e40 msgr2=0x7f0a4c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 -- 172.21.15.83:0/4243537778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c009550 con 0x7f0a4c0ca070 2023-11-28T11:08:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a50ff1700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 0x7f0a4c1e62a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a44007e60 tx=0x7f0a4400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400ed30 con 0x7f0a4c0ca070 2023-11-28T11:08:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400ee90 con 0x7f0a4c0ca070 2023-11-28T11:08:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44014660 con 0x7f0a4c0ca070 2023-11-28T11:08:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c06ad20 con 0x7f0a4c0ca070 2023-11-28T11:08:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.554+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c15ba00 con 0x7f0a4c0ca070 2023-11-28T11:08:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.555+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c06aeb0 con 0x7f0a4c0ca070 2023-11-28T11:08:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.556+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a44012050 con 0x7f0a4c0ca070 2023-11-28T11:08:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.557+0000 7f0a48ff9700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0a380682f0 0x7f0a3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.557+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0a44086f70 con 0x7f0a4c0ca070 2023-11-28T11:08:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.557+0000 7f0a4bfff700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0a380682f0 0x7f0a3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.557+0000 7f0a4bfff700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0a380682f0 0x7f0a3806a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a400045f0 tx=0x7f0a40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.560+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0a44056fb0 con 0x7f0a4c0ca070 2023-11-28T11:08:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.744+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a4c06e420 con 0x7f0a4c0ca070 2023-11-28T11:08:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.746+0000 7f0a48ff9700 1 -- 172.21.15.83:0/4243537778 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f0a44056ba0 con 0x7f0a4c0ca070 2023-11-28T11:08:49.747 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:49.747 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0a380682f0 msgr2=0x7f0a3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0a380682f0 0x7f0a3806a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a400045f0 tx=0x7f0a40008040 comp rx=0 tx=0).stop 2023-11-28T11:08:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 msgr2=0x7f0a4c1e62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 0x7f0a4c1e62a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a44007e60 tx=0x7f0a4400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 shutdown_connections 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0a380682f0 0x7f0a3806a7a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c0ca070 0x7f0a4c1e62a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0a4c06a950 0x7f0a4c1ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 --2- 172.21.15.83:0/4243537778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a4c0b4e40 0x7f0a4c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 >> 172.21.15.83:0/4243537778 conn(0x7f0a4c0cf4e0 msgr2=0x7f0a4c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 shutdown_connections 2023-11-28T11:08:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.748+0000 7f0a52a54700 1 -- 172.21.15.83:0/4243537778 wait complete. 2023-11-28T11:08:49.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.884+0000 7f053230c700 1 Processor -- start 2023-11-28T11:08:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.884+0000 7f053230c700 1 -- start start 2023-11-28T11:08:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f053230c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c15cd10 0x7f052c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f053230c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f052c0b1fb0 0x7f052c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f053230c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c0cbfb0 con 0x7f052c15d0e0 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f053230c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c06cee0 con 0x7f052c0b1fb0 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f053230c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0cb710 con 0x7f052c15cd10 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f052bfff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c15cd10 0x7f052c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f05308a9700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f052c0b1fb0 0x7f052c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f052b7fe700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f052c15d0e0 0x7f052c0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44790/0 (socket says 172.21.15.83:44790) 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f05308a9700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f052c0b1fb0 0x7f052c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35582/0 (socket says 172.21.15.83:35582) 2023-11-28T11:08:49.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f052b7fe700 1 -- 172.21.15.83:0/1344717547 learned_addr learned my addr 172.21.15.83:0/1344717547 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f05308a9700 1 -- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c15cd10 msgr2=0x7f052c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f05308a9700 1 --2- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c15cd10 0x7f052c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f05308a9700 1 -- 172.21.15.83:0/1344717547 >> v1:172.21.15.90:6789/0 conn(0x7f052c15d0e0 legacy=0x7f052c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f05308a9700 1 -- 172.21.15.83:0/1344717547 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0cb4e0 con 0x7f052c0b1fb0 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.885+0000 7f052affd700 1 -- 172.21.15.83:0/1344717547 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746353430 0 0) 0x7f052c0cbfb0 con 0x7f052c15d0e0 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f05308a9700 1 --2- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c0b1fb0 0x7f052c0cafa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0524009d90 tx=0x7f052400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a784a2d84bae5375 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f052affd700 1 -- 172.21.15.83:0/1344717547 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400e070 con 0x7f052c0b1fb0 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f052affd700 1 -- 172.21.15.83:0/1344717547 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052400bb20 con 0x7f052c0b1fb0 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f052affd700 1 -- 172.21.15.83:0/1344717547 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524012430 con 0x7f052c0b1fb0 2023-11-28T11:08:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 -- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c0b1fb0 msgr2=0x7f052c0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 --2- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c0b1fb0 0x7f052c0cafa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0524009d90 tx=0x7f052400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 -- 172.21.15.83:0/1344717547 shutdown_connections 2023-11-28T11:08:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 --2- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c0b1fb0 0x7f052c0cafa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 --2- 172.21.15.83:0/1344717547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c15cd10 0x7f052c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 -- 172.21.15.83:0/1344717547 >> 172.21.15.83:0/1344717547 conn(0x7f052c06e9a0 msgr2=0x7f052c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 -- 172.21.15.83:0/1344717547 shutdown_connections 2023-11-28T11:08:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.886+0000 7f053230c700 1 -- 172.21.15.83:0/1344717547 wait complete. 2023-11-28T11:08:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.888+0000 7f053230c700 1 Processor -- start 2023-11-28T11:08:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.888+0000 7f053230c700 1 -- start start 2023-11-28T11:08:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.888+0000 7f053230c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c0b1fb0 0x7f052c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.888+0000 7f053230c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c15cd10 0x7f052c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f053230c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 0x7f052c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f053230c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0b3520 con 0x7f052c15d0e0 2023-11-28T11:08:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f053230c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c6c00 con 0x7f052c15cd10 2023-11-28T11:08:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f053230c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c58b0 con 0x7f052c0b1fb0 2023-11-28T11:08:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 0x7f052c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 0x7f052c0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48592/0 (socket says 172.21.15.83:48592) 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 -- 172.21.15.83:0/2482743424 learned_addr learned my addr 172.21.15.83:0/2482743424 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f052b7fe700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c15cd10 0x7f052c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f052bfff700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c0b1fb0 0x7f052c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 -- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c0b1fb0 msgr2=0x7f052c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c0b1fb0 0x7f052c15c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 -- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c15cd10 msgr2=0x7f052c0c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c15cd10 0x7f052c0c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 -- 172.21.15.83:0/2482743424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009a70 con 0x7f052c15d0e0 2023-11-28T11:08:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f05308a9700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 0x7f052c0c5370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0524014040 tx=0x7f0524004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524012430 con 0x7f052c15d0e0 2023-11-28T11:08:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524020070 con 0x7f052c15d0e0 2023-11-28T11:08:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c0cbcd0 con 0x7f052c15d0e0 2023-11-28T11:08:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524004280 con 0x7f052c15d0e0 2023-11-28T11:08:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.889+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0cb9f0 con 0x7f052c15d0e0 2023-11-28T11:08:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.890+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c06b9b0 con 0x7f052c15d0e0 2023-11-28T11:08:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.891+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05240043e0 con 0x7f052c15d0e0 2023-11-28T11:08:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.891+0000 7f0528ff9700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f05180682f0 0x7f051806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.891+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0524022070 con 0x7f052c15d0e0 2023-11-28T11:08:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.891+0000 7f052bfff700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f05180682f0 0x7f051806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.892+0000 7f052bfff700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f05180682f0 0x7f051806a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f052c1e6b40 tx=0x7f0520009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:49.895+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0524054510 con 0x7f052c15d0e0 2023-11-28T11:08:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.079+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f052c0b3660 con 0x7f052c15d0e0 2023-11-28T11:08:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.080+0000 7f0528ff9700 1 -- 172.21.15.83:0/2482743424 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f052c0b3660 con 0x7f052c15d0e0 2023-11-28T11:08:50.082 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:50.082 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f05180682f0 msgr2=0x7f051806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f05180682f0 0x7f051806a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f052c1e6b40 tx=0x7f0520009210 comp rx=0 tx=0).stop 2023-11-28T11:08:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 msgr2=0x7f052c0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 0x7f052c0c5370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0524014040 tx=0x7f0524004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 shutdown_connections 2023-11-28T11:08:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f05180682f0 0x7f051806a7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052c15d0e0 0x7f052c0c5370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f052c0b1fb0 0x7f052c15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 --2- 172.21.15.83:0/2482743424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f052c15cd10 0x7f052c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 >> 172.21.15.83:0/2482743424 conn(0x7f052c06e9a0 msgr2=0x7f052c0b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 shutdown_connections 2023-11-28T11:08:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.082+0000 7f053230c700 1 -- 172.21.15.83:0/2482743424 wait complete. 2023-11-28T11:08:50.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:08:50.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.218+0000 7fb73f606700 1 Processor -- start 2023-11-28T11:08:50.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.218+0000 7fb73f606700 1 -- start start 2023-11-28T11:08:50.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73f606700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 0x7fb7380aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73f606700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73806d4a0 con 0x7fb7380b4bb0 2023-11-28T11:08:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73f606700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380b5ae0 con 0x7fb73815c890 2023-11-28T11:08:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73f606700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380b5050 con 0x7fb73806e370 2023-11-28T11:08:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73dba3700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7380b4bb0 0x7fb73806b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46460/0 (socket says 172.21.15.83:46460) 2023-11-28T11:08:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73dba3700 1 -- 172.21.15.83:0/2517600218 learned_addr learned my addr 172.21.15.83:0/2517600218 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb73cba1700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 0x7fb7380aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530724783 0 0) 0x7fb73806d4a0 con 0x7fb7380b4bb0 2023-11-28T11:08:50.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.219+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7240035b0 con 0x7fb7380b4bb0 2023-11-28T11:08:50.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272536446 0 0) 0x7fb7380b5ae0 con 0x7fb73815c890 2023-11-28T11:08:50.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb73806d4a0 con 0x7fb73815c890 2023-11-28T11:08:50.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640829968 0 0) 0x7fb7240035b0 con 0x7fb7380b4bb0 2023-11-28T11:08:50.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 msgr2=0x7fb7380aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 0x7fb7380aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 >> v1:172.21.15.90:6789/0 conn(0x7fb73815c890 legacy=0x7fb73806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73806c400 con 0x7fb7380b4bb0 2023-11-28T11:08:50.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fb7300019f0 con 0x7fb7380b4bb0 2023-11-28T11:08:50.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 >> v1:172.21.15.83:6789/0 conn(0x7fb7380b4bb0 legacy=0x7fb73806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb724003bc0 0x7fb724003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 0x7fb724004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72400bba0 con 0x7fb724003bc0 2023-11-28T11:08:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72400bce0 con 0x7fb724004d90 2023-11-28T11:08:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72400be50 con 0x7fb73806e370 2023-11-28T11:08:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb724003bc0 0x7fb724003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73cba1700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 0x7fb724004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73dba3700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 -- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 msgr2=0x7fb724004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb73806e370 0x7fb724004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 -- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724004d90 msgr2=0x7fb724005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 -- 172.21.15.83:0/2517600218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240035b0 con 0x7fb724003bc0 2023-11-28T11:08:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73dba3700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724004d90 0x7fb724005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb724003bc0 0x7fb724003fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb728001970 tx=0x7fb724005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdab26aee84891ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728001460 con 0x7fb724003bc0 2023-11-28T11:08:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7280052d0 con 0x7fb724003bc0 2023-11-28T11:08:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.220+0000 7fb72ffff700 1 -- 172.21.15.83:0/2517600218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728001da0 con 0x7fb724003bc0 2023-11-28T11:08:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73f606700 1 -- 172.21.15.83:0/2517600218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb724003bc0 msgr2=0x7fb724003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73f606700 1 --2- 172.21.15.83:0/2517600218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb724003bc0 0x7fb724003fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb728001970 tx=0x7fb724005880 comp rx=0 tx=0).stop 2023-11-28T11:08:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73dba3700 1 -- 172.21.15.83:0/2517600218 reap_dead start 2023-11-28T11:08:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73f606700 1 -- 172.21.15.83:0/2517600218 shutdown_connections 2023-11-28T11:08:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73f606700 1 -- 172.21.15.83:0/2517600218 >> 172.21.15.83:0/2517600218 conn(0x7fb7380b2b60 msgr2=0x7fb7380b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73f606700 1 -- 172.21.15.83:0/2517600218 shutdown_connections 2023-11-28T11:08:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.221+0000 7fb73f606700 1 -- 172.21.15.83:0/2517600218 wait complete. 2023-11-28T11:08:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.222+0000 7fb73f606700 1 Processor -- start 2023-11-28T11:08:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.222+0000 7fb73f606700 1 -- start start 2023-11-28T11:08:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73f606700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724003bc0 0x7fb7380c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73f606700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 0x7fb7380c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73f606700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb7380c8b00 0x7fb7380cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73f606700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380c92f0 con 0x7fb738154640 2023-11-28T11:08:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73f606700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380c7490 con 0x7fb724003bc0 2023-11-28T11:08:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73f606700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380c6e80 con 0x7fb7380c8b00 2023-11-28T11:08:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 0x7fb7380c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 0x7fb7380c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48610/0 (socket says 172.21.15.83:48610) 2023-11-28T11:08:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 -- 172.21.15.83:0/2843531786 learned_addr learned my addr 172.21.15.83:0/2843531786 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724003bc0 0x7fb7380c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73dba3700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb7380c8b00 0x7fb7380cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 -- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb7380c8b00 msgr2=0x7fb7380cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb7380c8b00 0x7fb7380cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 -- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724003bc0 msgr2=0x7fb7380c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724003bc0 0x7fb7380c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 -- 172.21.15.83:0/2843531786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380b5520 con 0x7fb738154640 2023-11-28T11:08:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.223+0000 7fb73cba1700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 0x7fb7380c85c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7340119f0 tx=0x7fb734011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.224+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011e90 con 0x7fb738154640 2023-11-28T11:08:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.224+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734004240 con 0x7fb738154640 2023-11-28T11:08:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.224+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7380b5240 con 0x7fb738154640 2023-11-28T11:08:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.224+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340036a0 con 0x7fb738154640 2023-11-28T11:08:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.224+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7380b5800 con 0x7fb738154640 2023-11-28T11:08:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.225+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7380c6cc0 con 0x7fb738154640 2023-11-28T11:08:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.226+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb734025070 con 0x7fb738154640 2023-11-28T11:08:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.226+0000 7fb72dffb700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb71c0682f0 0x7fb71c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.226+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb734086660 con 0x7fb738154640 2023-11-28T11:08:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.226+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb71c0682f0 0x7fb71c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.227+0000 7fb73d3a2700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb71c0682f0 0x7fb71c06a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb728008fd0 tx=0x7fb72800a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.230+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb734052bd0 con 0x7fb738154640 2023-11-28T11:08:50.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.457+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7380c75d0 con 0x7fb738154640 2023-11-28T11:08:50.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.825+0000 7fb72dffb700 1 -- 172.21.15.83:0/2843531786 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb7340561f0 con 0x7fb738154640 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb71c0682f0 msgr2=0x7fb71c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb71c0682f0 0x7fb71c06a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb728008fd0 tx=0x7fb72800a000 comp rx=0 tx=0).stop 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 msgr2=0x7fb7380c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 0x7fb7380c85c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7340119f0 tx=0x7fb734011d00 comp rx=0 tx=0).stop 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 shutdown_connections 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb7380c8b00 0x7fb7380cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb724003bc0 0x7fb7380c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb71c0682f0 0x7fb71c06a7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 --2- 172.21.15.83:0/2843531786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb738154640 0x7fb7380c85c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 >> 172.21.15.83:0/2843531786 conn(0x7fb7380b2b60 msgr2=0x7fb73806c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 shutdown_connections 2023-11-28T11:08:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.828+0000 7fb73f606700 1 -- 172.21.15.83:0/2843531786 wait complete. 2023-11-28T11:08:50.842 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-28T11:08:50.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:08:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 Processor -- start 2023-11-28T11:08:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 -- start start 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 0x7f7d0015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 0x7f7d000d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d00151a30 con 0x7f7d0015e5c0 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d000cb460 con 0x7f7d000ce1b0 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.965+0000 7f7d06daa700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000cb740 con 0x7f7d000d0ae0 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 0x7f7d000d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7cfffff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 0x7f7d0015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7cfffff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 0x7f7d0015b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35612/0 (socket says 172.21.15.83:35612) 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 0x7f7d000d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:32822/0 (socket says 172.21.15.83:32822) 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7cfffff700 1 -- 172.21.15.83:0/1019348938 learned_addr learned my addr 172.21.15.83:0/1019348938 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d04b46700 1 --1- 172.21.15.83:0/1019348938 >> v1:172.21.15.90:6789/0 conn(0x7f7d0015e5c0 0x7f7d0015e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44822/0 (socket says 172.21.15.83:44822) 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7cff7fe700 1 -- 172.21.15.83:0/1019348938 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083194687 0 0) 0x7f7d00151a30 con 0x7f7d0015e5c0 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7cff7fe700 1 -- 172.21.15.83:0/1019348938 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf00035b0 con 0x7f7d0015e5c0 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 -- 172.21.15.83:0/1019348938 >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 msgr2=0x7f7d0015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 --2- 172.21.15.83:0/1019348938 >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 0x7f7d0015b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 -- 172.21.15.83:0/1019348938 >> v1:172.21.15.90:6789/0 conn(0x7f7d0015e5c0 legacy=0x7f7d0015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 -- 172.21.15.83:0/1019348938 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000d1480 con 0x7f7d000d0ae0 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7cfffff700 1 --2- 172.21.15.83:0/1019348938 >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 0x7f7d0015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.966+0000 7f7d05347700 1 --2- 172.21.15.83:0/1019348938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 0x7f7d000d0f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009d90 tx=0x7f7cf800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a1241a36116ac8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7cff7fe700 1 -- 172.21.15.83:0/1019348938 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800e070 con 0x7f7d000d0ae0 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7cff7fe700 1 -- 172.21.15.83:0/1019348938 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf800bb20 con 0x7f7d000d0ae0 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7cff7fe700 1 -- 172.21.15.83:0/1019348938 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8012430 con 0x7f7d000d0ae0 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 -- 172.21.15.83:0/1019348938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 msgr2=0x7f7d000d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 --2- 172.21.15.83:0/1019348938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 0x7f7d000d0f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009d90 tx=0x7f7cf800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 -- 172.21.15.83:0/1019348938 shutdown_connections 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 --2- 172.21.15.83:0/1019348938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d000d0ae0 0x7f7d000d0f40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 --2- 172.21.15.83:0/1019348938 >> v2:172.21.15.90:3300/0 conn(0x7f7d000ce1b0 0x7f7d0015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 -- 172.21.15.83:0/1019348938 >> 172.21.15.83:0/1019348938 conn(0x7f7d0006ea50 msgr2=0x7f7d00070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 -- 172.21.15.83:0/1019348938 shutdown_connections 2023-11-28T11:08:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.967+0000 7f7d06daa700 1 -- 172.21.15.83:0/1019348938 wait complete. 2023-11-28T11:08:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.968+0000 7f7d06daa700 1 Processor -- start 2023-11-28T11:08:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.968+0000 7f7d06daa700 1 -- start start 2023-11-28T11:08:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d06daa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 0x7f7d00147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d06daa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d0015e5c0 0x7f7d000c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d06daa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d00147a20 0x7f7d00147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d06daa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d001478b0 con 0x7f7d000d0ae0 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d06daa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d001483e0 con 0x7f7d0015e5c0 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d06daa700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000c70e0 con 0x7f7d00147a20 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d04b46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 0x7f7d00147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d04b46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 0x7f7d00147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48622/0 (socket says 172.21.15.83:48622) 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d04b46700 1 -- 172.21.15.83:0/4088348796 learned_addr learned my addr 172.21.15.83:0/4088348796 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7cfffff700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d0015e5c0 0x7f7d000c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.969+0000 7f7d05347700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d00147a20 0x7f7d00147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d04b46700 1 -- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d00147a20 msgr2=0x7f7d00147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d04b46700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d00147a20 0x7f7d00147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d04b46700 1 -- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d0015e5c0 msgr2=0x7f7d000c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d04b46700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d0015e5c0 0x7f7d000c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d04b46700 1 -- 172.21.15.83:0/4088348796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec009550 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d04b46700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 0x7f7d00147370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40011e0 tx=0x7f7cf400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4005cb0 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4005e10 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8009a70 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40072f0 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.970+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0015ae90 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.971+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0015abb0 con 0x7f7d000d0ae0 2023-11-28T11:08:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.971+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7cf4019030 con 0x7f7d000d0ae0 2023-11-28T11:08:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.972+0000 7f7cfd7fa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ce40682f0 0x7f7ce406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.972+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7cf4085250 con 0x7f7d000d0ae0 2023-11-28T11:08:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.972+0000 7f7cfffff700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ce40682f0 0x7f7ce406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.972+0000 7f7cfffff700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ce40682f0 0x7f7ce406a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d000c7280 tx=0x7f7cec0092c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:50.976+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7cf4051730 con 0x7f7d000d0ae0 2023-11-28T11:08:51.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.206+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7d0015b8f0 con 0x7f7d000d0ae0 2023-11-28T11:08:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.834+0000 7f7cfd7fa700 1 -- 172.21.15.83:0/4088348796 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7cf4017020 con 0x7f7d000d0ae0 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ce40682f0 msgr2=0x7f7ce406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ce40682f0 0x7f7ce406a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d000c7280 tx=0x7f7cec0092c0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 msgr2=0x7f7d00147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 0x7f7d00147370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40011e0 tx=0x7f7cf400c9e0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 shutdown_connections 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7d00147a20 0x7f7d00147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ce40682f0 0x7f7ce406a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d000d0ae0 0x7f7d00147370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 --2- 172.21.15.83:0/4088348796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d0015e5c0 0x7f7d000c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 >> 172.21.15.83:0/4088348796 conn(0x7f7d000ce1b0 msgr2=0x7f7d000cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.837+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 shutdown_connections 2023-11-28T11:08:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.838+0000 7f7d06daa700 1 -- 172.21.15.83:0/4088348796 wait complete. 2023-11-28T11:08:51.854 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:08:51.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.978+0000 7f740cc6d700 1 Processor -- start 2023-11-28T11:08:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.978+0000 7f740cc6d700 1 -- start start 2023-11-28T11:08:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740cc6d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f74080ce3d0 0x7f740806b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740cc6d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f7408089480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740cc6d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7408169900 con 0x7f74080c9ad0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740cc6d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080ce7a0 con 0x7f74080ce3d0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740cc6d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740806a770 con 0x7f740806a270 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f7406d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f74080c9ad0 0x7f74080ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46494/0 (socket says 172.21.15.83:46494) 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f7406d9d700 1 -- 172.21.15.83:0/3529802157 learned_addr learned my addr 172.21.15.83:0/3529802157 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740659c700 1 --2- 172.21.15.83:0/3529802157 >> v2:172.21.15.90:3300/0 conn(0x7f74080ce3d0 0x7f740806b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f7405d9b700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f7408089480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:51.979+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064765373 0 0) 0x7f7400001490 con 0x7f74080c9ad0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f00034b0 con 0x7f74080c9ad0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2825979346 0 0) 0x7f73f00034b0 con 0x7f74080c9ad0 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 msgr2=0x7f7408089480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f7408089480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 >> v2:172.21.15.90:3300/0 conn(0x7f74080ce3d0 msgr2=0x7f740806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 --2- 172.21.15.83:0/3529802157 >> v2:172.21.15.90:3300/0 conn(0x7f74080ce3d0 0x7f740806b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740806aa50 con 0x7f74080c9ad0 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f7400001b80 con 0x7f74080c9ad0 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f7405d9b700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f7408089480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 >> v1:172.21.15.83:6789/0 conn(0x7f74080c9ad0 legacy=0x7f74080ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f0004ae0 0x7f73f0004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f00036f0 0x7f73f0003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f73f00040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080ce7a0 con 0x7f73f0004ae0 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740806a770 con 0x7f73f00036f0 2023-11-28T11:08:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f000fe40 con 0x7f740806a270 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740659c700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f0004ae0 0x7f73f0004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f7405d9b700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f00036f0 0x7f73f0003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f7406d9d700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f73f00040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740659c700 1 -- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 msgr2=0x7f73f00040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740659c700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f73f00040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740659c700 1 -- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f00036f0 msgr2=0x7f73f0003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740659c700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f00036f0 0x7f73f0003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f740659c700 1 -- 172.21.15.83:0/3529802157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f00034b0 con 0x7f73f0004ae0 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.010+0000 7f7406d9d700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f740806a270 0x7f73f00040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740659c700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f0004ae0 0x7f73f0004ef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73fc008a30 tx=0x7f73f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440d0cae594400c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc010040 con 0x7f73f0004ae0 2023-11-28T11:08:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc00b750 con 0x7f73f0004ae0 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740559a700 1 -- 172.21.15.83:0/3529802157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00bac0 con 0x7f73f0004ae0 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740cc6d700 1 -- 172.21.15.83:0/3529802157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f0004ae0 msgr2=0x7f73f0004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740cc6d700 1 --2- 172.21.15.83:0/3529802157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f0004ae0 0x7f73f0004ef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73fc008a30 tx=0x7f73f0004770 comp rx=0 tx=0).stop 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f7406d9d700 1 -- 172.21.15.83:0/3529802157 reap_dead start 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740cc6d700 1 -- 172.21.15.83:0/3529802157 shutdown_connections 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740cc6d700 1 -- 172.21.15.83:0/3529802157 >> 172.21.15.83:0/3529802157 conn(0x7f74080cf920 msgr2=0x7f74080cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740cc6d700 1 -- 172.21.15.83:0/3529802157 shutdown_connections 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.011+0000 7f740cc6d700 1 -- 172.21.15.83:0/3529802157 wait complete. 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 Processor -- start 2023-11-28T11:08:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 -- start start 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f0004ae0 0x7f740816bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74080c4da0 0x7f740816af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 0x7f74080b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740816c2a0 con 0x7f74080c45f0 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740816b4b0 con 0x7f73f0004ae0 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.013+0000 7f740cc6d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740816aae0 con 0x7f74080c4da0 2023-11-28T11:08:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 0x7f74080b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 0x7f74080b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48646/0 (socket says 172.21.15.83:48646) 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 -- 172.21.15.83:0/1842309673 learned_addr learned my addr 172.21.15.83:0/1842309673 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7405d9b700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74080c4da0 0x7f740816af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f740659c700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f0004ae0 0x7f740816bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 -- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74080c4da0 msgr2=0x7f740816af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74080c4da0 0x7f740816af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 -- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f0004ae0 msgr2=0x7f740816bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f0004ae0 0x7f740816bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 -- 172.21.15.83:0/1842309673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740806b010 con 0x7f74080c45f0 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7405d9b700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74080c4da0 0x7f740816af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f740659c700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f0004ae0 0x7f740816bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f7406d9d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 0x7f74080b1b60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f80 tx=0x7f740000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000ed10 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740000ee70 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740806ad30 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400013620 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.014+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740806b2f0 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.015+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740816ac50 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.016+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7400013780 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.016+0000 7f73f6ffd700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f73e80682f0 0x7f73e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.016+0000 7f740659c700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f73e80682f0 0x7f73e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.016+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7400086560 con 0x7f74080c45f0 2023-11-28T11:08:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.017+0000 7f740659c700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f73e80682f0 0x7f73e806a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74080ae960 tx=0x7f73fc012000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.020+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7400052ad0 con 0x7f74080c45f0 2023-11-28T11:08:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.245+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7408169900 con 0x7f74080c45f0 2023-11-28T11:08:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.245+0000 7f73f6ffd700 1 -- 172.21.15.83:0/1842309673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f7408169900 con 0x7f74080c45f0 2023-11-28T11:08:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:52.247 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:51.826077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f73e80682f0 msgr2=0x7f73e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f73e80682f0 0x7f73e806a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74080ae960 tx=0x7f73fc012000 comp rx=0 tx=0).stop 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 msgr2=0x7f74080b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 0x7f74080b1b60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f80 tx=0x7f740000e440 comp rx=0 tx=0).stop 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 shutdown_connections 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f73e80682f0 0x7f73e806a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74080c45f0 0x7f74080b1b60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73f0004ae0 0x7f740816bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 --2- 172.21.15.83:0/1842309673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74080c4da0 0x7f740816af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 >> 172.21.15.83:0/1842309673 conn(0x7f74080cf920 msgr2=0x7f74080ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 shutdown_connections 2023-11-28T11:08:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.247+0000 7f740cc6d700 1 -- 172.21.15.83:0/1842309673 wait complete. 2023-11-28T11:08:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2023-11-28T11:08:52.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.378+0000 7f3a1ea92700 1 Processor -- start 2023-11-28T11:08:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.378+0000 7f3a1ea92700 1 -- start start 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1ea92700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 0x7f3a180cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1ea92700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1806ca60 con 0x7f3a180b18a0 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1ea92700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180cf790 con 0x7f3a1806c5f0 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1ea92700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815d480 con 0x7f3a18159830 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1d02f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3a180b18a0 0x7f3a180cd9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46514/0 (socket says 172.21.15.83:46514) 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1d02f700 1 -- 172.21.15.83:0/728727072 learned_addr learned my addr 172.21.15.83:0/728727072 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.379+0000 7f3a1c82e700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 0x7f3a180cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075285428 0 0) 0x7f3a1806ca60 con 0x7f3a180b18a0 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a080035b0 con 0x7f3a180b18a0 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644153265 0 0) 0x7f3a080035b0 con 0x7f3a180b18a0 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 msgr2=0x7f3a180cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 0x7f3a180cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 >> v1:172.21.15.90:6789/0 conn(0x7f3a1806c5f0 legacy=0x7f3a1815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180cfa70 con 0x7f3a180b18a0 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f3a100019f0 con 0x7f3a180b18a0 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 >> v1:172.21.15.83:6789/0 conn(0x7f3a180b18a0 legacy=0x7f3a180cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 0x7f3a08003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a08003f70 0x7f3a08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a08005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815d480 con 0x7f3a08004d90 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0800ba40 con 0x7f3a08003f70 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0800bc00 con 0x7f3a18159830 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a08005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a17fff700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a08003f70 0x7f3a08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1c82e700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 0x7f3a08003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 -- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 msgr2=0x7f3a08003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18159830 0x7f3a08003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 -- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a08003f70 msgr2=0x7f3a08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a08003f70 0x7f3a08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 -- 172.21.15.83:0/728727072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080035b0 con 0x7f3a08004d90 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a1d02f700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a08005200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000a830 tx=0x7f3a08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c195c4fd99366aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.380+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10012600 con 0x7f3a08004d90 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10012780 con 0x7f3a08004d90 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a177fe700 1 -- 172.21.15.83:0/728727072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10012b30 con 0x7f3a08004d90 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1ea92700 1 -- 172.21.15.83:0/728727072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 msgr2=0x7f3a08005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/728727072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a08005200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000a830 tx=0x7f3a08005880 comp rx=0 tx=0).stop 2023-11-28T11:08:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1d02f700 1 -- 172.21.15.83:0/728727072 reap_dead start 2023-11-28T11:08:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1ea92700 1 -- 172.21.15.83:0/728727072 shutdown_connections 2023-11-28T11:08:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1ea92700 1 -- 172.21.15.83:0/728727072 >> 172.21.15.83:0/728727072 conn(0x7f3a1806e9a0 msgr2=0x7f3a180cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1ea92700 1 -- 172.21.15.83:0/728727072 shutdown_connections 2023-11-28T11:08:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.381+0000 7f3a1ea92700 1 -- 172.21.15.83:0/728727072 wait complete. 2023-11-28T11:08:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.382+0000 7f3a1ea92700 1 Processor -- start 2023-11-28T11:08:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.382+0000 7f3a1ea92700 1 -- start start 2023-11-28T11:08:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1ea92700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a1814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1ea92700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a1815c130 0x7f3a1815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1ea92700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18151030 0x7f3a181514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1ea92700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815cae0 con 0x7f3a08004d90 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1ea92700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1814f6c0 con 0x7f3a1815c130 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1ea92700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a181519e0 con 0x7f3a18151030 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1c82e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a1814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1c82e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a1814f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48662/0 (socket says 172.21.15.83:48662) 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1c82e700 1 -- 172.21.15.83:0/446534841 learned_addr learned my addr 172.21.15.83:0/446534841 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a17fff700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a1815c130 0x7f3a1815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.383+0000 7f3a1d02f700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18151030 0x7f3a181514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1c82e700 1 -- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18151030 msgr2=0x7f3a181514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1c82e700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18151030 0x7f3a181514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1c82e700 1 -- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a1815c130 msgr2=0x7f3a1815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1c82e700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a1815c130 0x7f3a1815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1c82e700 1 -- 172.21.15.83:0/446534841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1815d970 con 0x7f3a08004d90 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1c82e700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a1814f180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011aa0 tx=0x7f3a0c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00f980 con 0x7f3a08004d90 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00fae0 con 0x7f3a08004d90 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1815d690 con 0x7f3a08004d90 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00fe90 con 0x7f3a08004d90 2023-11-28T11:08:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.384+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a18159c00 con 0x7f3a08004d90 2023-11-28T11:08:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.385+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a180c89e0 con 0x7f3a08004d90 2023-11-28T11:08:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.385+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a0c014440 con 0x7f3a08004d90 2023-11-28T11:08:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.386+0000 7f3a157fa700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f39fc0682f0 0x7f39fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.386+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3a0c0869c0 con 0x7f3a08004d90 2023-11-28T11:08:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.386+0000 7f3a17fff700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f39fc0682f0 0x7f39fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.386+0000 7f3a17fff700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f39fc0682f0 0x7f39fc06a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a181f1350 tx=0x7f3a04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.390+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3a0c052ea0 con 0x7f3a08004d90 2023-11-28T11:08:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.577+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a18074320 con 0x7f3a08004d90 2023-11-28T11:08:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.578+0000 7f3a157fa700 1 -- 172.21.15.83:0/446534841 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f3a0c0564c0 con 0x7f3a08004d90 2023-11-28T11:08:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:52.580 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f39fc0682f0 msgr2=0x7f39fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f39fc0682f0 0x7f39fc06a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a181f1350 tx=0x7f3a04009040 comp rx=0 tx=0).stop 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 msgr2=0x7f3a1814f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a1814f180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011aa0 tx=0x7f3a0c011e60 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 shutdown_connections 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3a18151030 0x7f3a181514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f39fc0682f0 0x7f39fc06a7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a08004d90 0x7f3a1814f180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.580+0000 7f3a1ea92700 1 --2- 172.21.15.83:0/446534841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a1815c130 0x7f3a1815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.581+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 >> 172.21.15.83:0/446534841 conn(0x7f3a1806e9a0 msgr2=0x7f3a180b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.581+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 shutdown_connections 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.581+0000 7f3a1ea92700 1 -- 172.21.15.83:0/446534841 wait complete. 2023-11-28T11:08:52.594 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:08:52.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:52.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.714+0000 7f87590be700 1 Processor -- start 2023-11-28T11:08:52.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.714+0000 7f87590be700 1 -- start start 2023-11-28T11:08:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f87590be700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f87540ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f87590be700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f87540b1fb0 0x7f87540cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f87590be700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540cbfb0 con 0x7f875415d0e0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f87590be700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875406cee0 con 0x7f87540b1fb0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f87590be700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540cb710 con 0x7f875415cd10 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f875259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f875415d0e0 0x7f87540b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46530/0 (socket says 172.21.15.83:46530) 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f875259c700 1 -- 172.21.15.83:0/2011086056 learned_addr learned my addr 172.21.15.83:0/2011086056 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f875359e700 1 --2- 172.21.15.83:0/2011086056 >> v2:172.21.15.90:3300/0 conn(0x7f87540b1fb0 0x7f87540cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8752d9d700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f87540ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267495505 0 0) 0x7f87540cbfb0 con 0x7f875415d0e0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f873c0035b0 con 0x7f875415d0e0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3114473998 0 0) 0x7f873c0035b0 con 0x7f875415d0e0 2023-11-28T11:08:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 msgr2=0x7f87540ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f87540ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 >> v2:172.21.15.90:3300/0 conn(0x7f87540b1fb0 msgr2=0x7f87540cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 --2- 172.21.15.83:0/2011086056 >> v2:172.21.15.90:3300/0 conn(0x7f87540b1fb0 0x7f87540cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540cb4e0 con 0x7f875415d0e0 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f8744001670 con 0x7f875415d0e0 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 >> v1:172.21.15.83:6789/0 conn(0x7f875415d0e0 legacy=0x7f87540b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.715+0000 7f8751d9b700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f873c003a00 0x7f873c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f873c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f873c004d90 0x7f873c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875406cee0 con 0x7f873c003a00 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540cb710 con 0x7f873c004d90 2023-11-28T11:08:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c00ba10 con 0x7f875415cd10 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f873c003a00 0x7f873c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f875359e700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f873c004d90 0x7f873c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f875259c700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f873c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 -- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 msgr2=0x7f873c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f873c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 -- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f873c004d90 msgr2=0x7f873c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f873c004d90 0x7f873c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 -- 172.21.15.83:0/2011086056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0035b0 con 0x7f873c003a00 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f875259c700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f875415cd10 0x7f873c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8752d9d700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f873c003a00 0x7f873c003e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8748009b60 tx=0x7f873c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd62ac873cd2051f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800e660 con 0x7f873c003a00 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874800e7c0 con 0x7f873c003a00 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.716+0000 7f8751d9b700 1 -- 172.21.15.83:0/2011086056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800eb30 con 0x7f873c003a00 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f87590be700 1 -- 172.21.15.83:0/2011086056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f873c003a00 msgr2=0x7f873c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f87590be700 1 --2- 172.21.15.83:0/2011086056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f873c003a00 0x7f873c003e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8748009b60 tx=0x7f873c005880 comp rx=0 tx=0).stop 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f875359e700 1 -- 172.21.15.83:0/2011086056 reap_dead start 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f87590be700 1 -- 172.21.15.83:0/2011086056 shutdown_connections 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f87590be700 1 -- 172.21.15.83:0/2011086056 >> 172.21.15.83:0/2011086056 conn(0x7f875406e9a0 msgr2=0x7f87540cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f87590be700 1 -- 172.21.15.83:0/2011086056 shutdown_connections 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.717+0000 7f87590be700 1 -- 172.21.15.83:0/2011086056 wait complete. 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.718+0000 7f87590be700 1 Processor -- start 2023-11-28T11:08:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.718+0000 7f87590be700 1 -- start start 2023-11-28T11:08:52.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f87590be700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f873c003a00 0x7f8754160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f87590be700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 0x7f87540c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f87590be700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87540c6ef0 0x7f87540ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f87590be700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540c9180 con 0x7f87540ac2c0 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f87590be700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540c9610 con 0x7f87540c6ef0 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f87590be700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540c98f0 con 0x7f873c003a00 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 0x7f87540c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 0x7f87540c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48684/0 (socket says 172.21.15.83:48684) 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 -- 172.21.15.83:0/1561959627 learned_addr learned my addr 172.21.15.83:0/1561959627 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f8752d9d700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f873c003a00 0x7f8754160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875359e700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87540c6ef0 0x7f87540ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 -- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f873c003a00 msgr2=0x7f8754160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f873c003a00 0x7f8754160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 -- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87540c6ef0 msgr2=0x7f87540ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87540c6ef0 0x7f87540ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875259c700 1 -- 172.21.15.83:0/1561959627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540cbcd0 con 0x7f87540ac2c0 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.719+0000 7f875359e700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87540c6ef0 0x7f87540ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f8752d9d700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f873c003a00 0x7f8754160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f875259c700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 0x7f87540c8c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f874400eaf0 tx=0x7f874400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744005e00 con 0x7f87540ac2c0 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874400c7b0 con 0x7f87540ac2c0 2023-11-28T11:08:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87540cb9f0 con 0x7f87540ac2c0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400cb40 con 0x7f87540ac2c0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.720+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875406b9b0 con 0x7f87540ac2c0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.721+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8754074320 con 0x7f87540ac2c0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.721+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f874400cca0 con 0x7f87540ac2c0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.722+0000 7f873b7fe700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87400682f0 0x7f874006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.722+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8744085f30 con 0x7f87540ac2c0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.722+0000 7f8752d9d700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87400682f0 0x7f874006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.722+0000 7f8752d9d700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87400682f0 0x7f874006a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8748000c00 tx=0x7f8748008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.726+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87440524a0 con 0x7f87540ac2c0 2023-11-28T11:08:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.905+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87540c9780 con 0x7f87400682f0 2023-11-28T11:08:52.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.915+0000 7f873b7fe700 1 -- 172.21.15.83:0/1561959627 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367582 (secure 0 0 0) 0x7f87540c9780 con 0x7f87400682f0 2023-11-28T11:08:52.921 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87400682f0 msgr2=0x7f874006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87400682f0 0x7f874006a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8748000c00 tx=0x7f8748008030 comp rx=0 tx=0).stop 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 msgr2=0x7f87540c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 0x7f87540c8c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f874400eaf0 tx=0x7f874400eeb0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 shutdown_connections 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87540c6ef0 0x7f87540ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f873c003a00 0x7f8754160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87400682f0 0x7f874006a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 --2- 172.21.15.83:0/1561959627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87540ac2c0 0x7f87540c8c40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 >> 172.21.15.83:0/1561959627 conn(0x7f875406e9a0 msgr2=0x7f875415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 shutdown_connections 2023-11-28T11:08:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:52.919+0000 7f87590be700 1 -- 172.21.15.83:0/1561959627 wait complete. 2023-11-28T11:08:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:08:52.932 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-11-28T11:08:51.572164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594755,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67136,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907584,"statfs":{"total":773094113280,"available":773025366016,"internally_reserved":0,"allocated":3276800,"data_stored":2024628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2057,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.625161"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542681+0000","last_change":"2023-11-28T11:08:35.535687+0000","last_active":"2023-11-28T11:08:39.542681+0000","last_peered":"2023-11-28T11:08:39.542681+0000","last_clean":"2023-11-28T11:08:39.542681+0000","last_became_active":"2023-11-28T11:08:35.535025+0000","last_became_peered":"2023-11-28T11:08:35.535025+0000","last_unstale":"2023-11-28T11:08:39.542681+0000","last_undegraded":"2023-11-28T11:08:39.542681+0000","last_fullsized":"2023-11-28T11:08:39.542681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:21.578670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546994+0000","last_change":"2023-11-28T11:08:35.534288+0000","last_active":"2023-11-28T11:08:39.546994+0000","last_peered":"2023-11-28T11:08:39.546994+0000","last_clean":"2023-11-28T11:08:39.546994+0000","last_became_active":"2023-11-28T11:08:35.533997+0000","last_became_peered":"2023-11-28T11:08:35.533997+0000","last_unstale":"2023-11-28T11:08:39.546994+0000","last_undegraded":"2023-11-28T11:08:39.546994+0000","last_fullsized":"2023-11-28T11:08:39.546994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556945+0000","last_change":"2023-11-28T11:08:35.553042+0000","last_active":"2023-11-28T11:08:39.556945+0000","last_peered":"2023-11-28T11:08:39.556945+0000","last_clean":"2023-11-28T11:08:39.556945+0000","last_became_active":"2023-11-28T11:08:35.552693+0000","last_became_peered":"2023-11-28T11:08:35.552693+0000","last_unstale":"2023-11-28T11:08:39.556945+0000","last_undegraded":"2023-11-28T11:08:39.556945+0000","last_fullsized":"2023-11-28T11:08:39.556945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:43.204644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551967+0000","last_change":"2023-11-28T11:08:35.538585+0000","last_active":"2023-11-28T11:08:39.551967+0000","last_peered":"2023-11-28T11:08:39.551967+0000","last_clean":"2023-11-28T11:08:39.551967+0000","last_became_active":"2023-11-28T11:08:35.538074+0000","last_became_peered":"2023-11-28T11:08:35.538074+0000","last_unstale":"2023-11-28T11:08:39.551967+0000","last_undegraded":"2023-11-28T11:08:39.551967+0000","last_fullsized":"2023-11-28T11:08:39.551967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:13.796245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556389+0000","last_change":"2023-11-28T11:08:35.548649+0000","last_active":"2023-11-28T11:08:39.556389+0000","last_peered":"2023-11-28T11:08:39.556389+0000","last_clean":"2023-11-28T11:08:39.556389+0000","last_became_active":"2023-11-28T11:08:35.546646+0000","last_became_peered":"2023-11-28T11:08:35.546646+0000","last_unstale":"2023-11-28T11:08:39.556389+0000","last_undegraded":"2023-11-28T11:08:39.556389+0000","last_fullsized":"2023-11-28T11:08:39.556389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556449+0000","last_change":"2023-11-28T11:08:35.545320+0000","last_active":"2023-11-28T11:08:39.556449+0000","last_peered":"2023-11-28T11:08:39.556449+0000","last_clean":"2023-11-28T11:08:39.556449+0000","last_became_active":"2023-11-28T11:08:35.545051+0000","last_became_peered":"2023-11-28T11:08:35.545051+0000","last_unstale":"2023-11-28T11:08:39.556449+0000","last_undegraded":"2023-11-28T11:08:39.556449+0000","last_fullsized":"2023-11-28T11:08:39.556449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549589+0000","last_change":"2023-11-28T11:08:35.543641+0000","last_active":"2023-11-28T11:08:39.549589+0000","last_peered":"2023-11-28T11:08:39.549589+0000","last_clean":"2023-11-28T11:08:39.549589+0000","last_became_active":"2023-11-28T11:08:35.543407+0000","last_became_peered":"2023-11-28T11:08:35.543407+0000","last_unstale":"2023-11-28T11:08:39.549589+0000","last_undegraded":"2023-11-28T11:08:39.549589+0000","last_fullsized":"2023-11-28T11:08:39.549589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546097+0000","last_change":"2023-11-28T11:08:35.538922+0000","last_active":"2023-11-28T11:08:39.546097+0000","last_peered":"2023-11-28T11:08:39.546097+0000","last_clean":"2023-11-28T11:08:39.546097+0000","last_became_active":"2023-11-28T11:08:35.538455+0000","last_became_peered":"2023-11-28T11:08:35.538455+0000","last_unstale":"2023-11-28T11:08:39.546097+0000","last_undegraded":"2023-11-28T11:08:39.546097+0000","last_fullsized":"2023-11-28T11:08:39.546097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549757+0000","last_change":"2023-11-28T11:08:35.550345+0000","last_active":"2023-11-28T11:08:39.549757+0000","last_peered":"2023-11-28T11:08:39.549757+0000","last_clean":"2023-11-28T11:08:39.549757+0000","last_became_active":"2023-11-28T11:08:35.546222+0000","last_became_peered":"2023-11-28T11:08:35.546222+0000","last_unstale":"2023-11-28T11:08:39.549757+0000","last_undegraded":"2023-11-28T11:08:39.549757+0000","last_fullsized":"2023-11-28T11:08:39.549757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546977+0000","last_change":"2023-11-28T11:08:35.550152+0000","last_active":"2023-11-28T11:08:39.546977+0000","last_peered":"2023-11-28T11:08:39.546977+0000","last_clean":"2023-11-28T11:08:39.546977+0000","last_became_active":"2023-11-28T11:08:35.549618+0000","last_became_peered":"2023-11-28T11:08:35.549618+0000","last_unstale":"2023-11-28T11:08:39.546977+0000","last_undegraded":"2023-11-28T11:08:39.546977+0000","last_fullsized":"2023-11-28T11:08:39.546977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556901+0000","last_change":"2023-11-28T11:08:35.544292+0000","last_active":"2023-11-28T11:08:39.556901+0000","last_peered":"2023-11-28T11:08:39.556901+0000","last_clean":"2023-11-28T11:08:39.556901+0000","last_became_active":"2023-11-28T11:08:35.543905+0000","last_became_peered":"2023-11-28T11:08:35.543905+0000","last_unstale":"2023-11-28T11:08:39.556901+0000","last_undegraded":"2023-11-28T11:08:39.556901+0000","last_fullsized":"2023-11-28T11:08:39.556901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:46.079543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542701+0000","last_change":"2023-11-28T11:08:35.548959+0000","last_active":"2023-11-28T11:08:39.542701+0000","last_peered":"2023-11-28T11:08:39.542701+0000","last_clean":"2023-11-28T11:08:39.542701+0000","last_became_active":"2023-11-28T11:08:35.548707+0000","last_became_peered":"2023-11-28T11:08:35.548707+0000","last_unstale":"2023-11-28T11:08:39.542701+0000","last_undegraded":"2023-11-28T11:08:39.542701+0000","last_fullsized":"2023-11-28T11:08:39.542701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:41.426965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542575+0000","last_change":"2023-11-28T11:08:35.534783+0000","last_active":"2023-11-28T11:08:39.542575+0000","last_peered":"2023-11-28T11:08:39.542575+0000","last_clean":"2023-11-28T11:08:39.542575+0000","last_became_active":"2023-11-28T11:08:35.534520+0000","last_became_peered":"2023-11-28T11:08:35.534520+0000","last_unstale":"2023-11-28T11:08:39.542575+0000","last_undegraded":"2023-11-28T11:08:39.542575+0000","last_fullsized":"2023-11-28T11:08:39.542575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:56.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546312+0000","last_change":"2023-11-28T11:08:35.538322+0000","last_active":"2023-11-28T11:08:39.546312+0000","last_peered":"2023-11-28T11:08:39.546312+0000","last_clean":"2023-11-28T11:08:39.546312+0000","last_became_active":"2023-11-28T11:08:35.538000+0000","last_became_peered":"2023-11-28T11:08:35.538000+0000","last_unstale":"2023-11-28T11:08:39.546312+0000","last_undegraded":"2023-11-28T11:08:39.546312+0000","last_fullsized":"2023-11-28T11:08:39.546312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549399+0000","last_change":"2023-11-28T11:08:35.544771+0000","last_active":"2023-11-28T11:08:39.549399+0000","last_peered":"2023-11-28T11:08:39.549399+0000","last_clean":"2023-11-28T11:08:39.549399+0000","last_became_active":"2023-11-28T11:08:35.544525+0000","last_became_peered":"2023-11-28T11:08:35.544525+0000","last_unstale":"2023-11-28T11:08:39.549399+0000","last_undegraded":"2023-11-28T11:08:39.549399+0000","last_fullsized":"2023-11-28T11:08:39.549399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552566+0000","last_change":"2023-11-28T11:08:35.547686+0000","last_active":"2023-11-28T11:08:39.552566+0000","last_peered":"2023-11-28T11:08:39.552566+0000","last_clean":"2023-11-28T11:08:39.552566+0000","last_became_active":"2023-11-28T11:08:35.547411+0000","last_became_peered":"2023-11-28T11:08:35.547411+0000","last_unstale":"2023-11-28T11:08:39.552566+0000","last_undegraded":"2023-11-28T11:08:39.552566+0000","last_fullsized":"2023-11-28T11:08:39.552566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549830+0000","last_change":"2023-11-28T11:08:35.551541+0000","last_active":"2023-11-28T11:08:39.549830+0000","last_peered":"2023-11-28T11:08:39.549830+0000","last_clean":"2023-11-28T11:08:39.549830+0000","last_became_active":"2023-11-28T11:08:35.546691+0000","last_became_peered":"2023-11-28T11:08:35.546691+0000","last_unstale":"2023-11-28T11:08:39.549830+0000","last_undegraded":"2023-11-28T11:08:39.549830+0000","last_fullsized":"2023-11-28T11:08:39.549830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556352+0000","last_change":"2023-11-28T11:08:35.541127+0000","last_active":"2023-11-28T11:08:39.556352+0000","last_peered":"2023-11-28T11:08:39.556352+0000","last_clean":"2023-11-28T11:08:39.556352+0000","last_became_active":"2023-11-28T11:08:35.540800+0000","last_became_peered":"2023-11-28T11:08:35.540800+0000","last_unstale":"2023-11-28T11:08:39.556352+0000","last_undegraded":"2023-11-28T11:08:39.556352+0000","last_fullsized":"2023-11-28T11:08:39.556352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.543580+0000","last_change":"2023-11-28T11:08:35.546695+0000","last_active":"2023-11-28T11:08:39.543580+0000","last_peered":"2023-11-28T11:08:39.543580+0000","last_clean":"2023-11-28T11:08:39.543580+0000","last_became_active":"2023-11-28T11:08:35.546438+0000","last_became_peered":"2023-11-28T11:08:35.546438+0000","last_unstale":"2023-11-28T11:08:39.543580+0000","last_undegraded":"2023-11-28T11:08:39.543580+0000","last_fullsized":"2023-11-28T11:08:39.543580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:37.007416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556699+0000","last_change":"2023-11-28T11:08:35.548019+0000","last_active":"2023-11-28T11:08:39.556699+0000","last_peered":"2023-11-28T11:08:39.556699+0000","last_clean":"2023-11-28T11:08:39.556699+0000","last_became_active":"2023-11-28T11:08:35.541157+0000","last_became_peered":"2023-11-28T11:08:35.541157+0000","last_unstale":"2023-11-28T11:08:39.556699+0000","last_undegraded":"2023-11-28T11:08:39.556699+0000","last_fullsized":"2023-11-28T11:08:39.556699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:40.346506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551701+0000","last_change":"2023-11-28T11:08:35.540145+0000","last_active":"2023-11-28T11:08:39.551701+0000","last_peered":"2023-11-28T11:08:39.551701+0000","last_clean":"2023-11-28T11:08:39.551701+0000","last_became_active":"2023-11-28T11:08:35.539880+0000","last_became_peered":"2023-11-28T11:08:35.539880+0000","last_unstale":"2023-11-28T11:08:39.551701+0000","last_undegraded":"2023-11-28T11:08:39.551701+0000","last_fullsized":"2023-11-28T11:08:39.551701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549283+0000","last_change":"2023-11-28T11:08:35.546609+0000","last_active":"2023-11-28T11:08:39.549283+0000","last_peered":"2023-11-28T11:08:39.549283+0000","last_clean":"2023-11-28T11:08:39.549283+0000","last_became_active":"2023-11-28T11:08:35.545637+0000","last_became_peered":"2023-11-28T11:08:35.545637+0000","last_unstale":"2023-11-28T11:08:39.549283+0000","last_undegraded":"2023-11-28T11:08:39.549283+0000","last_fullsized":"2023-11-28T11:08:39.549283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544746+0000","last_change":"2023-11-28T11:08:35.540494+0000","last_active":"2023-11-28T11:08:39.544746+0000","last_peered":"2023-11-28T11:08:39.544746+0000","last_clean":"2023-11-28T11:08:39.544746+0000","last_became_active":"2023-11-28T11:08:35.539936+0000","last_became_peered":"2023-11-28T11:08:35.539936+0000","last_unstale":"2023-11-28T11:08:39.544746+0000","last_undegraded":"2023-11-28T11:08:39.544746+0000","last_fullsized":"2023-11-28T11:08:39.544746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550772+0000","last_change":"2023-11-28T11:08:35.545757+0000","last_active":"2023-11-28T11:08:39.550772+0000","last_peered":"2023-11-28T11:08:39.550772+0000","last_clean":"2023-11-28T11:08:39.550772+0000","last_became_active":"2023-11-28T11:08:35.545459+0000","last_became_peered":"2023-11-28T11:08:35.545459+0000","last_unstale":"2023-11-28T11:08:39.550772+0000","last_undegraded":"2023-11-28T11:08:39.550772+0000","last_fullsized":"2023-11-28T11:08:39.550772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556095+0000","last_change":"2023-11-28T11:08:35.540552+0000","last_active":"2023-11-28T11:08:39.556095+0000","last_peered":"2023-11-28T11:08:39.556095+0000","last_clean":"2023-11-28T11:08:39.556095+0000","last_became_active":"2023-11-28T11:08:35.539972+0000","last_became_peered":"2023-11-28T11:08:35.539972+0000","last_unstale":"2023-11-28T11:08:39.556095+0000","last_undegraded":"2023-11-28T11:08:39.556095+0000","last_fullsized":"2023-11-28T11:08:39.556095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556230+0000","last_change":"2023-11-28T11:08:35.536499+0000","last_active":"2023-11-28T11:08:39.556230+0000","last_peered":"2023-11-28T11:08:39.556230+0000","last_clean":"2023-11-28T11:08:39.556230+0000","last_became_active":"2023-11-28T11:08:35.536308+0000","last_became_peered":"2023-11-28T11:08:35.536308+0000","last_unstale":"2023-11-28T11:08:39.556230+0000","last_undegraded":"2023-11-28T11:08:39.556230+0000","last_fullsized":"2023-11-28T11:08:39.556230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546678+0000","last_change":"2023-11-28T11:08:35.542161+0000","last_active":"2023-11-28T11:08:39.546678+0000","last_peered":"2023-11-28T11:08:39.546678+0000","last_clean":"2023-11-28T11:08:39.546678+0000","last_became_active":"2023-11-28T11:08:35.541331+0000","last_became_peered":"2023-11-28T11:08:35.541331+0000","last_unstale":"2023-11-28T11:08:39.546678+0000","last_undegraded":"2023-11-28T11:08:39.546678+0000","last_fullsized":"2023-11-28T11:08:39.546678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570777+0000","last_change":"2023-11-28T11:08:39.566823+0000","last_active":"2023-11-28T11:08:39.570777+0000","last_peered":"2023-11-28T11:08:39.570777+0000","last_clean":"2023-11-28T11:08:39.570777+0000","last_became_active":"2023-11-28T11:08:39.563462+0000","last_became_peered":"2023-11-28T11:08:39.563462+0000","last_unstale":"2023-11-28T11:08:39.570777+0000","last_undegraded":"2023-11-28T11:08:39.570777+0000","last_fullsized":"2023-11-28T11:08:39.570777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:25.821485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556318+0000","last_change":"2023-11-28T11:08:35.144920+0000","last_active":"2023-11-28T11:08:39.556318+0000","last_peered":"2023-11-28T11:08:39.556318+0000","last_clean":"2023-11-28T11:08:39.556318+0000","last_became_active":"2023-11-28T11:08:35.144360+0000","last_became_peered":"2023-11-28T11:08:35.144360+0000","last_unstale":"2023-11-28T11:08:39.556318+0000","last_undegraded":"2023-11-28T11:08:39.556318+0000","last_fullsized":"2023-11-28T11:08:39.556318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544509+0000","last_change":"2023-11-28T11:08:35.537580+0000","last_active":"2023-11-28T11:08:39.544509+0000","last_peered":"2023-11-28T11:08:39.544509+0000","last_clean":"2023-11-28T11:08:39.544509+0000","last_became_active":"2023-11-28T11:08:35.537300+0000","last_became_peered":"2023-11-28T11:08:35.537300+0000","last_unstale":"2023-11-28T11:08:39.544509+0000","last_undegraded":"2023-11-28T11:08:39.544509+0000","last_fullsized":"2023-11-28T11:08:39.544509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573888+0000","last_change":"2023-11-28T11:08:39.565761+0000","last_active":"2023-11-28T11:08:39.573888+0000","last_peered":"2023-11-28T11:08:39.573888+0000","last_clean":"2023-11-28T11:08:39.573888+0000","last_became_active":"2023-11-28T11:08:39.565387+0000","last_became_peered":"2023-11-28T11:08:39.565387+0000","last_unstale":"2023-11-28T11:08:39.573888+0000","last_undegraded":"2023-11-28T11:08:39.573888+0000","last_fullsized":"2023-11-28T11:08:39.573888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:29.648074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552645+0000","last_change":"2023-11-28T11:08:35.141536+0000","last_active":"2023-11-28T11:08:39.552645+0000","last_peered":"2023-11-28T11:08:39.552645+0000","last_clean":"2023-11-28T11:08:39.552645+0000","last_became_active":"2023-11-28T11:08:35.141259+0000","last_became_peered":"2023-11-28T11:08:35.141259+0000","last_unstale":"2023-11-28T11:08:39.552645+0000","last_undegraded":"2023-11-28T11:08:39.552645+0000","last_fullsized":"2023-11-28T11:08:39.552645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551925+0000","last_change":"2023-11-28T11:08:35.551235+0000","last_active":"2023-11-28T11:08:39.551925+0000","last_peered":"2023-11-28T11:08:39.551925+0000","last_clean":"2023-11-28T11:08:39.551925+0000","last_became_active":"2023-11-28T11:08:35.550945+0000","last_became_peered":"2023-11-28T11:08:35.550945+0000","last_unstale":"2023-11-28T11:08:39.551925+0000","last_undegraded":"2023-11-28T11:08:39.551925+0000","last_fullsized":"2023-11-28T11:08:39.551925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.576247+0000","last_change":"2023-11-28T11:08:39.574297+0000","last_active":"2023-11-28T11:08:39.576247+0000","last_peered":"2023-11-28T11:08:39.576247+0000","last_clean":"2023-11-28T11:08:39.576247+0000","last_became_active":"2023-11-28T11:08:39.573836+0000","last_became_peered":"2023-11-28T11:08:39.573836+0000","last_unstale":"2023-11-28T11:08:39.576247+0000","last_undegraded":"2023-11-28T11:08:39.576247+0000","last_fullsized":"2023-11-28T11:08:39.576247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546095+0000","last_change":"2023-11-28T11:08:35.140807+0000","last_active":"2023-11-28T11:08:39.546095+0000","last_peered":"2023-11-28T11:08:39.546095+0000","last_clean":"2023-11-28T11:08:39.546095+0000","last_became_active":"2023-11-28T11:08:35.140551+0000","last_became_peered":"2023-11-28T11:08:35.140551+0000","last_unstale":"2023-11-28T11:08:39.546095+0000","last_undegraded":"2023-11-28T11:08:39.546095+0000","last_fullsized":"2023-11-28T11:08:39.546095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553480+0000","last_change":"2023-11-28T11:08:35.540497+0000","last_active":"2023-11-28T11:08:39.553480+0000","last_peered":"2023-11-28T11:08:39.553480+0000","last_clean":"2023-11-28T11:08:39.553480+0000","last_became_active":"2023-11-28T11:08:35.540154+0000","last_became_peered":"2023-11-28T11:08:35.540154+0000","last_unstale":"2023-11-28T11:08:39.553480+0000","last_undegraded":"2023-11-28T11:08:39.553480+0000","last_fullsized":"2023-11-28T11:08:39.553480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:43.354051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.574001+0000","last_change":"2023-11-28T11:08:39.568636+0000","last_active":"2023-11-28T11:08:39.574001+0000","last_peered":"2023-11-28T11:08:39.574001+0000","last_clean":"2023-11-28T11:08:39.574001+0000","last_became_active":"2023-11-28T11:08:39.568255+0000","last_became_peered":"2023-11-28T11:08:39.568255+0000","last_unstale":"2023-11-28T11:08:39.574001+0000","last_undegraded":"2023-11-28T11:08:39.574001+0000","last_fullsized":"2023-11-28T11:08:39.574001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549903+0000","last_change":"2023-11-28T11:08:35.139702+0000","last_active":"2023-11-28T11:08:39.549903+0000","last_peered":"2023-11-28T11:08:39.549903+0000","last_clean":"2023-11-28T11:08:39.549903+0000","last_became_active":"2023-11-28T11:08:35.139379+0000","last_became_peered":"2023-11-28T11:08:35.139379+0000","last_unstale":"2023-11-28T11:08:39.549903+0000","last_undegraded":"2023-11-28T11:08:39.549903+0000","last_fullsized":"2023-11-28T11:08:39.549903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546007+0000","last_change":"2023-11-28T11:08:35.532733+0000","last_active":"2023-11-28T11:08:39.546007+0000","last_peered":"2023-11-28T11:08:39.546007+0000","last_clean":"2023-11-28T11:08:39.546007+0000","last_became_active":"2023-11-28T11:08:35.532390+0000","last_became_peered":"2023-11-28T11:08:35.532390+0000","last_unstale":"2023-11-28T11:08:39.546007+0000","last_undegraded":"2023-11-28T11:08:39.546007+0000","last_fullsized":"2023-11-28T11:08:39.546007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573440+0000","last_change":"2023-11-28T11:08:39.568834+0000","last_active":"2023-11-28T11:08:39.573440+0000","last_peered":"2023-11-28T11:08:39.573440+0000","last_clean":"2023-11-28T11:08:39.573440+0000","last_became_active":"2023-11-28T11:08:39.568153+0000","last_became_peered":"2023-11-28T11:08:39.568153+0000","last_unstale":"2023-11-28T11:08:39.573440+0000","last_undegraded":"2023-11-28T11:08:39.573440+0000","last_fullsized":"2023-11-28T11:08:39.573440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546860+0000","last_change":"2023-11-28T11:08:35.140772+0000","last_active":"2023-11-28T11:08:39.546860+0000","last_peered":"2023-11-28T11:08:39.546860+0000","last_clean":"2023-11-28T11:08:39.546860+0000","last_became_active":"2023-11-28T11:08:35.140418+0000","last_became_peered":"2023-11-28T11:08:35.140418+0000","last_unstale":"2023-11-28T11:08:39.546860+0000","last_undegraded":"2023-11-28T11:08:39.546860+0000","last_fullsized":"2023-11-28T11:08:39.546860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544363+0000","last_change":"2023-11-28T11:08:35.544651+0000","last_active":"2023-11-28T11:08:39.544363+0000","last_peered":"2023-11-28T11:08:39.544363+0000","last_clean":"2023-11-28T11:08:39.544363+0000","last_became_active":"2023-11-28T11:08:35.541280+0000","last_became_peered":"2023-11-28T11:08:35.541280+0000","last_unstale":"2023-11-28T11:08:39.544363+0000","last_undegraded":"2023-11-28T11:08:39.544363+0000","last_fullsized":"2023-11-28T11:08:39.544363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568222+0000","last_change":"2023-11-28T11:08:39.562858+0000","last_active":"2023-11-28T11:08:39.568222+0000","last_peered":"2023-11-28T11:08:39.568222+0000","last_clean":"2023-11-28T11:08:39.568222+0000","last_became_active":"2023-11-28T11:08:39.562273+0000","last_became_peered":"2023-11-28T11:08:39.562273+0000","last_unstale":"2023-11-28T11:08:39.568222+0000","last_undegraded":"2023-11-28T11:08:39.568222+0000","last_fullsized":"2023-11-28T11:08:39.568222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:39.831045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552124+0000","last_change":"2023-11-28T11:08:35.144557+0000","last_active":"2023-11-28T11:08:39.552124+0000","last_peered":"2023-11-28T11:08:39.552124+0000","last_clean":"2023-11-28T11:08:39.552124+0000","last_became_active":"2023-11-28T11:08:35.144222+0000","last_became_peered":"2023-11-28T11:08:35.144222+0000","last_unstale":"2023-11-28T11:08:39.552124+0000","last_undegraded":"2023-11-28T11:08:39.552124+0000","last_fullsized":"2023-11-28T11:08:39.552124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552994+0000","last_change":"2023-11-28T11:08:35.543394+0000","last_active":"2023-11-28T11:08:39.552994+0000","last_peered":"2023-11-28T11:08:39.552994+0000","last_clean":"2023-11-28T11:08:39.552994+0000","last_became_active":"2023-11-28T11:08:35.543099+0000","last_became_peered":"2023-11-28T11:08:35.543099+0000","last_unstale":"2023-11-28T11:08:39.552994+0000","last_undegraded":"2023-11-28T11:08:39.552994+0000","last_fullsized":"2023-11-28T11:08:39.552994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571358+0000","last_change":"2023-11-28T11:08:39.570301+0000","last_active":"2023-11-28T11:08:39.571358+0000","last_peered":"2023-11-28T11:08:39.571358+0000","last_clean":"2023-11-28T11:08:39.571358+0000","last_became_active":"2023-11-28T11:08:39.569828+0000","last_became_peered":"2023-11-28T11:08:39.569828+0000","last_unstale":"2023-11-28T11:08:39.571358+0000","last_undegraded":"2023-11-28T11:08:39.571358+0000","last_fullsized":"2023-11-28T11:08:39.571358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:43.778416+0000","last_change":"2023-11-28T11:08:35.147185+0000","last_active":"2023-11-28T11:08:43.778416+0000","last_peered":"2023-11-28T11:08:43.778416+0000","last_clean":"2023-11-28T11:08:43.778416+0000","last_became_active":"2023-11-28T11:08:35.146285+0000","last_became_peered":"2023-11-28T11:08:35.146285+0000","last_unstale":"2023-11-28T11:08:43.778416+0000","last_undegraded":"2023-11-28T11:08:43.778416+0000","last_fullsized":"2023-11-28T11:08:43.778416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3605,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556565+0000","last_change":"2023-11-28T11:08:35.546895+0000","last_active":"2023-11-28T11:08:39.556565+0000","last_peered":"2023-11-28T11:08:39.556565+0000","last_clean":"2023-11-28T11:08:39.556565+0000","last_became_active":"2023-11-28T11:08:35.538447+0000","last_became_peered":"2023-11-28T11:08:35.538447+0000","last_unstale":"2023-11-28T11:08:39.556565+0000","last_undegraded":"2023-11-28T11:08:39.556565+0000","last_fullsized":"2023-11-28T11:08:39.556565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:28.436759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564343+0000","last_change":"2023-11-28T11:08:39.560554+0000","last_active":"2023-11-28T11:08:39.564343+0000","last_peered":"2023-11-28T11:08:39.564343+0000","last_clean":"2023-11-28T11:08:39.564343+0000","last_became_active":"2023-11-28T11:08:39.560032+0000","last_became_peered":"2023-11-28T11:08:39.560032+0000","last_unstale":"2023-11-28T11:08:39.564343+0000","last_undegraded":"2023-11-28T11:08:39.564343+0000","last_fullsized":"2023-11-28T11:08:39.564343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551526+0000","last_change":"2023-11-28T11:08:35.146846+0000","last_active":"2023-11-28T11:08:39.551526+0000","last_peered":"2023-11-28T11:08:39.551526+0000","last_clean":"2023-11-28T11:08:39.551526+0000","last_became_active":"2023-11-28T11:08:35.146565+0000","last_became_peered":"2023-11-28T11:08:35.146565+0000","last_unstale":"2023-11-28T11:08:39.551526+0000","last_undegraded":"2023-11-28T11:08:39.551526+0000","last_fullsized":"2023-11-28T11:08:39.551526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545959+0000","last_change":"2023-11-28T11:08:35.537655+0000","last_active":"2023-11-28T11:08:39.545959+0000","last_peered":"2023-11-28T11:08:39.545959+0000","last_clean":"2023-11-28T11:08:39.545959+0000","last_became_active":"2023-11-28T11:08:35.537401+0000","last_became_peered":"2023-11-28T11:08:35.537401+0000","last_unstale":"2023-11-28T11:08:39.545959+0000","last_undegraded":"2023-11-28T11:08:39.545959+0000","last_fullsized":"2023-11-28T11:08:39.545959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.559490+0000","last_change":"2023-11-28T11:08:39.558756+0000","last_active":"2023-11-28T11:08:39.559490+0000","last_peered":"2023-11-28T11:08:39.559490+0000","last_clean":"2023-11-28T11:08:39.559490+0000","last_became_active":"2023-11-28T11:08:39.558382+0000","last_became_peered":"2023-11-28T11:08:39.558382+0000","last_unstale":"2023-11-28T11:08:39.559490+0000","last_undegraded":"2023-11-28T11:08:39.559490+0000","last_fullsized":"2023-11-28T11:08:39.559490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:08.685769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549379+0000","last_change":"2023-11-28T11:08:35.143155+0000","last_active":"2023-11-28T11:08:39.549379+0000","last_peered":"2023-11-28T11:08:39.549379+0000","last_clean":"2023-11-28T11:08:39.549379+0000","last_became_active":"2023-11-28T11:08:35.142856+0000","last_became_peered":"2023-11-28T11:08:35.142856+0000","last_unstale":"2023-11-28T11:08:39.549379+0000","last_undegraded":"2023-11-28T11:08:39.549379+0000","last_fullsized":"2023-11-28T11:08:39.549379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546595+0000","last_change":"2023-11-28T11:08:35.550010+0000","last_active":"2023-11-28T11:08:39.546595+0000","last_peered":"2023-11-28T11:08:39.546595+0000","last_clean":"2023-11-28T11:08:39.546595+0000","last_became_active":"2023-11-28T11:08:35.548764+0000","last_became_peered":"2023-11-28T11:08:35.548764+0000","last_unstale":"2023-11-28T11:08:39.546595+0000","last_undegraded":"2023-11-28T11:08:39.546595+0000","last_fullsized":"2023-11-28T11:08:39.546595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568175+0000","last_change":"2023-11-28T11:08:39.562607+0000","last_active":"2023-11-28T11:08:39.568175+0000","last_peered":"2023-11-28T11:08:39.568175+0000","last_clean":"2023-11-28T11:08:39.568175+0000","last_became_active":"2023-11-28T11:08:39.562144+0000","last_became_peered":"2023-11-28T11:08:39.562144+0000","last_unstale":"2023-11-28T11:08:39.568175+0000","last_undegraded":"2023-11-28T11:08:39.568175+0000","last_fullsized":"2023-11-28T11:08:39.568175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:43:13.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549488+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.549488+0000","last_peered":"2023-11-28T11:08:39.549488+0000","last_clean":"2023-11-28T11:08:39.549488+0000","last_became_active":"2023-11-28T11:08:35.146468+0000","last_became_peered":"2023-11-28T11:08:35.146468+0000","last_unstale":"2023-11-28T11:08:39.549488+0000","last_undegraded":"2023-11-28T11:08:39.549488+0000","last_fullsized":"2023-11-28T11:08:39.549488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:42.348852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552146+0000","last_change":"2023-11-28T11:08:35.545831+0000","last_active":"2023-11-28T11:08:39.552146+0000","last_peered":"2023-11-28T11:08:39.552146+0000","last_clean":"2023-11-28T11:08:39.552146+0000","last_became_active":"2023-11-28T11:08:35.544509+0000","last_became_peered":"2023-11-28T11:08:35.544509+0000","last_unstale":"2023-11-28T11:08:39.552146+0000","last_undegraded":"2023-11-28T11:08:39.552146+0000","last_fullsized":"2023-11-28T11:08:39.552146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564184+0000","last_change":"2023-11-28T11:08:39.560957+0000","last_active":"2023-11-28T11:08:39.564184+0000","last_peered":"2023-11-28T11:08:39.564184+0000","last_clean":"2023-11-28T11:08:39.564184+0000","last_became_active":"2023-11-28T11:08:39.560011+0000","last_became_peered":"2023-11-28T11:08:39.560011+0000","last_unstale":"2023-11-28T11:08:39.564184+0000","last_undegraded":"2023-11-28T11:08:39.564184+0000","last_fullsized":"2023-11-28T11:08:39.564184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551361+0000","last_change":"2023-11-28T11:08:35.137486+0000","last_active":"2023-11-28T11:08:39.551361+0000","last_peered":"2023-11-28T11:08:39.551361+0000","last_clean":"2023-11-28T11:08:39.551361+0000","last_became_active":"2023-11-28T11:08:35.137192+0000","last_became_peered":"2023-11-28T11:08:35.137192+0000","last_unstale":"2023-11-28T11:08:39.551361+0000","last_undegraded":"2023-11-28T11:08:39.551361+0000","last_fullsized":"2023-11-28T11:08:39.551361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:09.971307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545264+0000","last_change":"2023-11-28T11:08:35.543758+0000","last_active":"2023-11-28T11:08:39.545264+0000","last_peered":"2023-11-28T11:08:39.545264+0000","last_clean":"2023-11-28T11:08:39.545264+0000","last_became_active":"2023-11-28T11:08:35.543535+0000","last_became_peered":"2023-11-28T11:08:35.543535+0000","last_unstale":"2023-11-28T11:08:39.545264+0000","last_undegraded":"2023-11-28T11:08:39.545264+0000","last_fullsized":"2023-11-28T11:08:39.545264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570639+0000","last_change":"2023-11-28T11:08:39.567742+0000","last_active":"2023-11-28T11:08:39.570639+0000","last_peered":"2023-11-28T11:08:39.570639+0000","last_clean":"2023-11-28T11:08:39.570639+0000","last_became_active":"2023-11-28T11:08:39.567313+0000","last_became_peered":"2023-11-28T11:08:39.567313+0000","last_unstale":"2023-11-28T11:08:39.570639+0000","last_undegraded":"2023-11-28T11:08:39.570639+0000","last_fullsized":"2023-11-28T11:08:39.570639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552255+0000","last_change":"2023-11-28T11:08:35.142255+0000","last_active":"2023-11-28T11:08:39.552255+0000","last_peered":"2023-11-28T11:08:39.552255+0000","last_clean":"2023-11-28T11:08:39.552255+0000","last_became_active":"2023-11-28T11:08:35.140243+0000","last_became_peered":"2023-11-28T11:08:35.140243+0000","last_unstale":"2023-11-28T11:08:39.552255+0000","last_undegraded":"2023-11-28T11:08:39.552255+0000","last_fullsized":"2023-11-28T11:08:39.552255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:43.134431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546061+0000","last_change":"2023-11-28T11:08:35.540595+0000","last_active":"2023-11-28T11:08:39.546061+0000","last_peered":"2023-11-28T11:08:39.546061+0000","last_clean":"2023-11-28T11:08:39.546061+0000","last_became_active":"2023-11-28T11:08:35.540303+0000","last_became_peered":"2023-11-28T11:08:35.540303+0000","last_unstale":"2023-11-28T11:08:39.546061+0000","last_undegraded":"2023-11-28T11:08:39.546061+0000","last_fullsized":"2023-11-28T11:08:39.546061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562667+0000","last_change":"2023-11-28T11:08:39.560601+0000","last_active":"2023-11-28T11:08:39.562667+0000","last_peered":"2023-11-28T11:08:39.562667+0000","last_clean":"2023-11-28T11:08:39.562667+0000","last_became_active":"2023-11-28T11:08:39.560147+0000","last_became_peered":"2023-11-28T11:08:39.560147+0000","last_unstale":"2023-11-28T11:08:39.562667+0000","last_undegraded":"2023-11-28T11:08:39.562667+0000","last_fullsized":"2023-11-28T11:08:39.562667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:39.607355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552185+0000","last_change":"2023-11-28T11:08:35.142804+0000","last_active":"2023-11-28T11:08:39.552185+0000","last_peered":"2023-11-28T11:08:39.552185+0000","last_clean":"2023-11-28T11:08:39.552185+0000","last_became_active":"2023-11-28T11:08:35.142407+0000","last_became_peered":"2023-11-28T11:08:35.142407+0000","last_unstale":"2023-11-28T11:08:39.552185+0000","last_undegraded":"2023-11-28T11:08:39.552185+0000","last_fullsized":"2023-11-28T11:08:39.552185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:12.791046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560104+0000","last_change":"2023-11-28T11:08:35.543448+0000","last_active":"2023-11-28T11:08:39.560104+0000","last_peered":"2023-11-28T11:08:39.560104+0000","last_clean":"2023-11-28T11:08:39.560104+0000","last_became_active":"2023-11-28T11:08:35.541094+0000","last_became_peered":"2023-11-28T11:08:35.541094+0000","last_unstale":"2023-11-28T11:08:39.560104+0000","last_undegraded":"2023-11-28T11:08:39.560104+0000","last_fullsized":"2023-11-28T11:08:39.560104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571651+0000","last_change":"2023-11-28T11:08:39.562935+0000","last_active":"2023-11-28T11:08:39.571651+0000","last_peered":"2023-11-28T11:08:39.571651+0000","last_clean":"2023-11-28T11:08:39.571651+0000","last_became_active":"2023-11-28T11:08:39.562529+0000","last_became_peered":"2023-11-28T11:08:39.562529+0000","last_unstale":"2023-11-28T11:08:39.571651+0000","last_undegraded":"2023-11-28T11:08:39.571651+0000","last_fullsized":"2023-11-28T11:08:39.571651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:31.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552244+0000","last_change":"2023-11-28T11:08:35.143035+0000","last_active":"2023-11-28T11:08:39.552244+0000","last_peered":"2023-11-28T11:08:39.552244+0000","last_clean":"2023-11-28T11:08:39.552244+0000","last_became_active":"2023-11-28T11:08:35.142697+0000","last_became_peered":"2023-11-28T11:08:35.142697+0000","last_unstale":"2023-11-28T11:08:39.552244+0000","last_undegraded":"2023-11-28T11:08:39.552244+0000","last_fullsized":"2023-11-28T11:08:39.552244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:24.895027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552639+0000","last_change":"2023-11-28T11:08:35.535083+0000","last_active":"2023-11-28T11:08:39.552639+0000","last_peered":"2023-11-28T11:08:39.552639+0000","last_clean":"2023-11-28T11:08:39.552639+0000","last_became_active":"2023-11-28T11:08:35.534800+0000","last_became_peered":"2023-11-28T11:08:35.534800+0000","last_unstale":"2023-11-28T11:08:39.552639+0000","last_undegraded":"2023-11-28T11:08:39.552639+0000","last_fullsized":"2023-11-28T11:08:39.552639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571669+0000","last_change":"2023-11-28T11:08:39.565208+0000","last_active":"2023-11-28T11:08:39.571669+0000","last_peered":"2023-11-28T11:08:39.571669+0000","last_clean":"2023-11-28T11:08:39.571669+0000","last_became_active":"2023-11-28T11:08:39.564299+0000","last_became_peered":"2023-11-28T11:08:39.564299+0000","last_unstale":"2023-11-28T11:08:39.571669+0000","last_undegraded":"2023-11-28T11:08:39.571669+0000","last_fullsized":"2023-11-28T11:08:39.571669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:48.486752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546343+0000","last_change":"2023-11-28T11:08:35.141725+0000","last_active":"2023-11-28T11:08:39.546343+0000","last_peered":"2023-11-28T11:08:39.546343+0000","last_clean":"2023-11-28T11:08:39.546343+0000","last_became_active":"2023-11-28T11:08:35.141445+0000","last_became_peered":"2023-11-28T11:08:35.141445+0000","last_unstale":"2023-11-28T11:08:39.546343+0000","last_undegraded":"2023-11-28T11:08:39.546343+0000","last_fullsized":"2023-11-28T11:08:39.546343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556057+0000","last_change":"2023-11-28T11:08:35.548819+0000","last_active":"2023-11-28T11:08:39.556057+0000","last_peered":"2023-11-28T11:08:39.556057+0000","last_clean":"2023-11-28T11:08:39.556057+0000","last_became_active":"2023-11-28T11:08:35.548403+0000","last_became_peered":"2023-11-28T11:08:35.548403+0000","last_unstale":"2023-11-28T11:08:39.556057+0000","last_undegraded":"2023-11-28T11:08:39.556057+0000","last_fullsized":"2023-11-28T11:08:39.556057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571605+0000","last_change":"2023-11-28T11:08:39.562941+0000","last_active":"2023-11-28T11:08:39.571605+0000","last_peered":"2023-11-28T11:08:39.571605+0000","last_clean":"2023-11-28T11:08:39.571605+0000","last_became_active":"2023-11-28T11:08:39.562555+0000","last_became_peered":"2023-11-28T11:08:39.562555+0000","last_unstale":"2023-11-28T11:08:39.571605+0000","last_undegraded":"2023-11-28T11:08:39.571605+0000","last_fullsized":"2023-11-28T11:08:39.571605+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544942+0000","last_change":"2023-11-28T11:08:35.142283+0000","last_active":"2023-11-28T11:08:39.544942+0000","last_peered":"2023-11-28T11:08:39.544942+0000","last_clean":"2023-11-28T11:08:39.544942+0000","last_became_active":"2023-11-28T11:08:35.141976+0000","last_became_peered":"2023-11-28T11:08:35.141976+0000","last_unstale":"2023-11-28T11:08:39.544942+0000","last_undegraded":"2023-11-28T11:08:39.544942+0000","last_fullsized":"2023-11-28T11:08:39.544942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556613+0000","last_change":"2023-11-28T11:08:35.540882+0000","last_active":"2023-11-28T11:08:39.556613+0000","last_peered":"2023-11-28T11:08:39.556613+0000","last_clean":"2023-11-28T11:08:39.556613+0000","last_became_active":"2023-11-28T11:08:35.540566+0000","last_became_peered":"2023-11-28T11:08:35.540566+0000","last_unstale":"2023-11-28T11:08:39.556613+0000","last_undegraded":"2023-11-28T11:08:39.556613+0000","last_fullsized":"2023-11-28T11:08:39.556613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570782+0000","last_change":"2023-11-28T11:08:39.566349+0000","last_active":"2023-11-28T11:08:39.570782+0000","last_peered":"2023-11-28T11:08:39.570782+0000","last_clean":"2023-11-28T11:08:39.570782+0000","last_became_active":"2023-11-28T11:08:39.565923+0000","last_became_peered":"2023-11-28T11:08:39.565923+0000","last_unstale":"2023-11-28T11:08:39.570782+0000","last_undegraded":"2023-11-28T11:08:39.570782+0000","last_fullsized":"2023-11-28T11:08:39.570782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551243+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.551243+0000","last_peered":"2023-11-28T11:08:39.551243+0000","last_clean":"2023-11-28T11:08:39.551243+0000","last_became_active":"2023-11-28T11:08:35.146466+0000","last_became_peered":"2023-11-28T11:08:35.146466+0000","last_unstale":"2023-11-28T11:08:39.551243+0000","last_undegraded":"2023-11-28T11:08:39.551243+0000","last_fullsized":"2023-11-28T11:08:39.551243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556761+0000","last_change":"2023-11-28T11:08:35.540325+0000","last_active":"2023-11-28T11:08:39.556761+0000","last_peered":"2023-11-28T11:08:39.556761+0000","last_clean":"2023-11-28T11:08:39.556761+0000","last_became_active":"2023-11-28T11:08:35.540045+0000","last_became_peered":"2023-11-28T11:08:35.540045+0000","last_unstale":"2023-11-28T11:08:39.556761+0000","last_undegraded":"2023-11-28T11:08:39.556761+0000","last_fullsized":"2023-11-28T11:08:39.556761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566338+0000","last_change":"2023-11-28T11:08:39.562874+0000","last_active":"2023-11-28T11:08:39.566338+0000","last_peered":"2023-11-28T11:08:39.566338+0000","last_clean":"2023-11-28T11:08:39.566338+0000","last_became_active":"2023-11-28T11:08:39.562484+0000","last_became_peered":"2023-11-28T11:08:39.562484+0000","last_unstale":"2023-11-28T11:08:39.566338+0000","last_undegraded":"2023-11-28T11:08:39.566338+0000","last_fullsized":"2023-11-28T11:08:39.566338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552660+0000","last_change":"2023-11-28T11:08:35.139637+0000","last_active":"2023-11-28T11:08:39.552660+0000","last_peered":"2023-11-28T11:08:39.552660+0000","last_clean":"2023-11-28T11:08:39.552660+0000","last_became_active":"2023-11-28T11:08:35.138268+0000","last_became_peered":"2023-11-28T11:08:35.138268+0000","last_unstale":"2023-11-28T11:08:39.552660+0000","last_undegraded":"2023-11-28T11:08:39.552660+0000","last_fullsized":"2023-11-28T11:08:39.552660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556539+0000","last_change":"2023-11-28T11:08:35.543314+0000","last_active":"2023-11-28T11:08:39.556539+0000","last_peered":"2023-11-28T11:08:39.556539+0000","last_clean":"2023-11-28T11:08:39.556539+0000","last_became_active":"2023-11-28T11:08:35.539838+0000","last_became_peered":"2023-11-28T11:08:35.539838+0000","last_unstale":"2023-11-28T11:08:39.556539+0000","last_undegraded":"2023-11-28T11:08:39.556539+0000","last_fullsized":"2023-11-28T11:08:39.556539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:53.884226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568846+0000","last_change":"2023-11-28T11:08:39.565918+0000","last_active":"2023-11-28T11:08:39.568846+0000","last_peered":"2023-11-28T11:08:39.568846+0000","last_clean":"2023-11-28T11:08:39.568846+0000","last_became_active":"2023-11-28T11:08:39.562911+0000","last_became_peered":"2023-11-28T11:08:39.562911+0000","last_unstale":"2023-11-28T11:08:39.568846+0000","last_undegraded":"2023-11-28T11:08:39.568846+0000","last_fullsized":"2023-11-28T11:08:39.568846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:11:09.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546326+0000","last_change":"2023-11-28T11:08:35.143397+0000","last_active":"2023-11-28T11:08:39.546326+0000","last_peered":"2023-11-28T11:08:39.546326+0000","last_clean":"2023-11-28T11:08:39.546326+0000","last_became_active":"2023-11-28T11:08:35.142466+0000","last_became_peered":"2023-11-28T11:08:35.142466+0000","last_unstale":"2023-11-28T11:08:39.546326+0000","last_undegraded":"2023-11-28T11:08:39.546326+0000","last_fullsized":"2023-11-28T11:08:39.546326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556168+0000","last_change":"2023-11-28T11:08:35.534626+0000","last_active":"2023-11-28T11:08:39.556168+0000","last_peered":"2023-11-28T11:08:39.556168+0000","last_clean":"2023-11-28T11:08:39.556168+0000","last_became_active":"2023-11-28T11:08:35.534379+0000","last_became_peered":"2023-11-28T11:08:35.534379+0000","last_unstale":"2023-11-28T11:08:39.556168+0000","last_undegraded":"2023-11-28T11:08:39.556168+0000","last_fullsized":"2023-11-28T11:08:39.556168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573390+0000","last_change":"2023-11-28T11:08:39.569661+0000","last_active":"2023-11-28T11:08:39.573390+0000","last_peered":"2023-11-28T11:08:39.573390+0000","last_clean":"2023-11-28T11:08:39.573390+0000","last_became_active":"2023-11-28T11:08:39.569175+0000","last_became_peered":"2023-11-28T11:08:39.569175+0000","last_unstale":"2023-11-28T11:08:39.573390+0000","last_undegraded":"2023-11-28T11:08:39.573390+0000","last_fullsized":"2023-11-28T11:08:39.573390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:59.446120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544982+0000","last_change":"2023-11-28T11:08:35.143399+0000","last_active":"2023-11-28T11:08:39.544982+0000","last_peered":"2023-11-28T11:08:39.544982+0000","last_clean":"2023-11-28T11:08:39.544982+0000","last_became_active":"2023-11-28T11:08:35.143102+0000","last_became_peered":"2023-11-28T11:08:35.143102+0000","last_unstale":"2023-11-28T11:08:39.544982+0000","last_undegraded":"2023-11-28T11:08:39.544982+0000","last_fullsized":"2023-11-28T11:08:39.544982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556842+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:08:39.556842+0000","last_peered":"2023-11-28T11:08:39.556842+0000","last_clean":"2023-11-28T11:08:39.556842+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:08:39.556842+0000","last_undegraded":"2023-11-28T11:08:39.556842+0000","last_fullsized":"2023-11-28T11:08:39.556842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545915+0000","last_change":"2023-11-28T11:08:35.541344+0000","last_active":"2023-11-28T11:08:39.545915+0000","last_peered":"2023-11-28T11:08:39.545915+0000","last_clean":"2023-11-28T11:08:39.545915+0000","last_became_active":"2023-11-28T11:08:35.541086+0000","last_became_peered":"2023-11-28T11:08:35.541086+0000","last_unstale":"2023-11-28T11:08:39.545915+0000","last_undegraded":"2023-11-28T11:08:39.545915+0000","last_fullsized":"2023-11-28T11:08:39.545915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.572142+0000","last_change":"2023-11-28T11:08:39.567333+0000","last_active":"2023-11-28T11:08:39.572142+0000","last_peered":"2023-11-28T11:08:39.572142+0000","last_clean":"2023-11-28T11:08:39.572142+0000","last_became_active":"2023-11-28T11:08:39.566952+0000","last_became_peered":"2023-11-28T11:08:39.566952+0000","last_unstale":"2023-11-28T11:08:39.572142+0000","last_undegraded":"2023-11-28T11:08:39.572142+0000","last_fullsized":"2023-11-28T11:08:39.572142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545875+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:08:39.545875+0000","last_peered":"2023-11-28T11:08:39.545875+0000","last_clean":"2023-11-28T11:08:39.545875+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:08:39.545875+0000","last_undegraded":"2023-11-28T11:08:39.545875+0000","last_fullsized":"2023-11-28T11:08:39.545875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:40.959549+0000","last_change":"2023-11-28T11:08:35.138175+0000","last_active":"2023-11-28T11:08:40.959549+0000","last_peered":"2023-11-28T11:08:40.959549+0000","last_clean":"2023-11-28T11:08:40.959549+0000","last_became_active":"2023-11-28T11:08:35.137577+0000","last_became_peered":"2023-11-28T11:08:35.137577+0000","last_unstale":"2023-11-28T11:08:40.959549+0000","last_undegraded":"2023-11-28T11:08:40.959549+0000","last_fullsized":"2023-11-28T11:08:40.959549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560017+0000","last_change":"2023-11-28T11:08:35.540521+0000","last_active":"2023-11-28T11:08:39.560017+0000","last_peered":"2023-11-28T11:08:39.560017+0000","last_clean":"2023-11-28T11:08:39.560017+0000","last_became_active":"2023-11-28T11:08:35.538206+0000","last_became_peered":"2023-11-28T11:08:35.538206+0000","last_unstale":"2023-11-28T11:08:39.560017+0000","last_undegraded":"2023-11-28T11:08:39.560017+0000","last_fullsized":"2023-11-28T11:08:39.560017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568578+0000","last_change":"2023-11-28T11:08:39.565755+0000","last_active":"2023-11-28T11:08:39.568578+0000","last_peered":"2023-11-28T11:08:39.568578+0000","last_clean":"2023-11-28T11:08:39.568578+0000","last_became_active":"2023-11-28T11:08:39.565093+0000","last_became_peered":"2023-11-28T11:08:39.565093+0000","last_unstale":"2023-11-28T11:08:39.568578+0000","last_undegraded":"2023-11-28T11:08:39.568578+0000","last_fullsized":"2023-11-28T11:08:39.568578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555643+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:08:39.555643+0000","last_peered":"2023-11-28T11:08:39.555643+0000","last_clean":"2023-11-28T11:08:39.555643+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:08:39.555643+0000","last_undegraded":"2023-11-28T11:08:39.555643+0000","last_fullsized":"2023-11-28T11:08:39.555643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550860+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:08:39.550860+0000","last_peered":"2023-11-28T11:08:39.550860+0000","last_clean":"2023-11-28T11:08:39.550860+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:08:39.550860+0000","last_undegraded":"2023-11-28T11:08:39.550860+0000","last_fullsized":"2023-11-28T11:08:39.550860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549175+0000","last_change":"2023-11-28T11:08:35.146578+0000","last_active":"2023-11-28T11:08:39.549175+0000","last_peered":"2023-11-28T11:08:39.549175+0000","last_clean":"2023-11-28T11:08:39.549175+0000","last_became_active":"2023-11-28T11:08:35.146300+0000","last_became_peered":"2023-11-28T11:08:35.146300+0000","last_unstale":"2023-11-28T11:08:39.549175+0000","last_undegraded":"2023-11-28T11:08:39.549175+0000","last_fullsized":"2023-11-28T11:08:39.549175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552070+0000","last_change":"2023-11-28T11:08:35.545787+0000","last_active":"2023-11-28T11:08:39.552070+0000","last_peered":"2023-11-28T11:08:39.552070+0000","last_clean":"2023-11-28T11:08:39.552070+0000","last_became_active":"2023-11-28T11:08:35.544359+0000","last_became_peered":"2023-11-28T11:08:35.544359+0000","last_unstale":"2023-11-28T11:08:39.552070+0000","last_undegraded":"2023-11-28T11:08:39.552070+0000","last_fullsized":"2023-11-28T11:08:39.552070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.561183+0000","last_change":"2023-11-28T11:08:39.558516+0000","last_active":"2023-11-28T11:08:39.561183+0000","last_peered":"2023-11-28T11:08:39.561183+0000","last_clean":"2023-11-28T11:08:39.561183+0000","last_became_active":"2023-11-28T11:08:39.558117+0000","last_became_peered":"2023-11-28T11:08:39.558117+0000","last_unstale":"2023-11-28T11:08:39.561183+0000","last_undegraded":"2023-11-28T11:08:39.561183+0000","last_fullsized":"2023-11-28T11:08:39.561183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551550+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:08:39.551550+0000","last_peered":"2023-11-28T11:08:39.551550+0000","last_clean":"2023-11-28T11:08:39.551550+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:08:39.551550+0000","last_undegraded":"2023-11-28T11:08:39.551550+0000","last_fullsized":"2023-11-28T11:08:39.551550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.554459+0000","last_change":"2023-11-28T11:08:35.134990+0000","last_active":"2023-11-28T11:08:39.554459+0000","last_peered":"2023-11-28T11:08:39.554459+0000","last_clean":"2023-11-28T11:08:39.554459+0000","last_became_active":"2023-11-28T11:08:35.134728+0000","last_became_peered":"2023-11-28T11:08:35.134728+0000","last_unstale":"2023-11-28T11:08:39.554459+0000","last_undegraded":"2023-11-28T11:08:39.554459+0000","last_fullsized":"2023-11-28T11:08:39.554459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556117+0000","last_change":"2023-11-28T11:08:35.540730+0000","last_active":"2023-11-28T11:08:39.556117+0000","last_peered":"2023-11-28T11:08:39.556117+0000","last_clean":"2023-11-28T11:08:39.556117+0000","last_became_active":"2023-11-28T11:08:35.540312+0000","last_became_peered":"2023-11-28T11:08:35.540312+0000","last_unstale":"2023-11-28T11:08:39.556117+0000","last_undegraded":"2023-11-28T11:08:39.556117+0000","last_fullsized":"2023-11-28T11:08:39.556117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568532+0000","last_change":"2023-11-28T11:08:39.561818+0000","last_active":"2023-11-28T11:08:39.568532+0000","last_peered":"2023-11-28T11:08:39.568532+0000","last_clean":"2023-11-28T11:08:39.568532+0000","last_became_active":"2023-11-28T11:08:39.561292+0000","last_became_peered":"2023-11-28T11:08:39.561292+0000","last_unstale":"2023-11-28T11:08:39.568532+0000","last_undegraded":"2023-11-28T11:08:39.568532+0000","last_fullsized":"2023-11-28T11:08:39.568532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550685+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:08:39.550685+0000","last_peered":"2023-11-28T11:08:39.550685+0000","last_clean":"2023-11-28T11:08:39.550685+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:08:39.550685+0000","last_undegraded":"2023-11-28T11:08:39.550685+0000","last_fullsized":"2023-11-28T11:08:39.550685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546311+0000","last_change":"2023-11-28T11:08:35.140783+0000","last_active":"2023-11-28T11:08:39.546311+0000","last_peered":"2023-11-28T11:08:39.546311+0000","last_clean":"2023-11-28T11:08:39.546311+0000","last_became_active":"2023-11-28T11:08:35.140512+0000","last_became_peered":"2023-11-28T11:08:35.140512+0000","last_unstale":"2023-11-28T11:08:39.546311+0000","last_undegraded":"2023-11-28T11:08:39.546311+0000","last_fullsized":"2023-11-28T11:08:39.546311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551352+0000","last_change":"2023-11-28T11:08:35.547824+0000","last_active":"2023-11-28T11:08:39.551352+0000","last_peered":"2023-11-28T11:08:39.551352+0000","last_clean":"2023-11-28T11:08:39.551352+0000","last_became_active":"2023-11-28T11:08:35.547517+0000","last_became_peered":"2023-11-28T11:08:35.547517+0000","last_unstale":"2023-11-28T11:08:39.551352+0000","last_undegraded":"2023-11-28T11:08:39.551352+0000","last_fullsized":"2023-11-28T11:08:39.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:55.715375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570503+0000","last_change":"2023-11-28T11:08:39.564906+0000","last_active":"2023-11-28T11:08:39.570503+0000","last_peered":"2023-11-28T11:08:39.570503+0000","last_clean":"2023-11-28T11:08:39.570503+0000","last_became_active":"2023-11-28T11:08:39.564258+0000","last_became_peered":"2023-11-28T11:08:39.564258+0000","last_unstale":"2023-11-28T11:08:39.570503+0000","last_undegraded":"2023-11-28T11:08:39.570503+0000","last_fullsized":"2023-11-28T11:08:39.570503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546086+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:08:39.546086+0000","last_peered":"2023-11-28T11:08:39.546086+0000","last_clean":"2023-11-28T11:08:39.546086+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:08:39.546086+0000","last_undegraded":"2023-11-28T11:08:39.546086+0000","last_fullsized":"2023-11-28T11:08:39.546086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556375+0000","last_change":"2023-11-28T11:08:35.144005+0000","last_active":"2023-11-28T11:08:39.556375+0000","last_peered":"2023-11-28T11:08:39.556375+0000","last_clean":"2023-11-28T11:08:39.556375+0000","last_became_active":"2023-11-28T11:08:35.141190+0000","last_became_peered":"2023-11-28T11:08:35.141190+0000","last_unstale":"2023-11-28T11:08:39.556375+0000","last_undegraded":"2023-11-28T11:08:39.556375+0000","last_fullsized":"2023-11-28T11:08:39.556375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555324+0000","last_change":"2023-11-28T11:08:35.539604+0000","last_active":"2023-11-28T11:08:39.555324+0000","last_peered":"2023-11-28T11:08:39.555324+0000","last_clean":"2023-11-28T11:08:39.555324+0000","last_became_active":"2023-11-28T11:08:35.539319+0000","last_became_peered":"2023-11-28T11:08:35.539319+0000","last_unstale":"2023-11-28T11:08:39.555324+0000","last_undegraded":"2023-11-28T11:08:39.555324+0000","last_fullsized":"2023-11-28T11:08:39.555324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562105+0000","last_change":"2023-11-28T11:08:39.559157+0000","last_active":"2023-11-28T11:08:39.562105+0000","last_peered":"2023-11-28T11:08:39.562105+0000","last_clean":"2023-11-28T11:08:39.562105+0000","last_became_active":"2023-11-28T11:08:39.558790+0000","last_became_peered":"2023-11-28T11:08:39.558790+0000","last_unstale":"2023-11-28T11:08:39.562105+0000","last_undegraded":"2023-11-28T11:08:39.562105+0000","last_fullsized":"2023-11-28T11:08:39.562105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:18.469803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550458+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:08:39.550458+0000","last_peered":"2023-11-28T11:08:39.550458+0000","last_clean":"2023-11-28T11:08:39.550458+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:08:39.550458+0000","last_undegraded":"2023-11-28T11:08:39.550458+0000","last_fullsized":"2023-11-28T11:08:39.550458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555493+0000","last_change":"2023-11-28T11:08:35.136293+0000","last_active":"2023-11-28T11:08:39.555493+0000","last_peered":"2023-11-28T11:08:39.555493+0000","last_clean":"2023-11-28T11:08:39.555493+0000","last_became_active":"2023-11-28T11:08:35.136011+0000","last_became_peered":"2023-11-28T11:08:35.136011+0000","last_unstale":"2023-11-28T11:08:39.555493+0000","last_undegraded":"2023-11-28T11:08:39.555493+0000","last_fullsized":"2023-11-28T11:08:39.555493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556312+0000","last_change":"2023-11-28T11:08:35.548876+0000","last_active":"2023-11-28T11:08:39.556312+0000","last_peered":"2023-11-28T11:08:39.556312+0000","last_clean":"2023-11-28T11:08:39.556312+0000","last_became_active":"2023-11-28T11:08:35.548606+0000","last_became_peered":"2023-11-28T11:08:35.548606+0000","last_unstale":"2023-11-28T11:08:39.556312+0000","last_undegraded":"2023-11-28T11:08:39.556312+0000","last_fullsized":"2023-11-28T11:08:39.556312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570495+0000","last_change":"2023-11-28T11:08:39.564356+0000","last_active":"2023-11-28T11:08:39.570495+0000","last_peered":"2023-11-28T11:08:39.570495+0000","last_clean":"2023-11-28T11:08:39.570495+0000","last_became_active":"2023-11-28T11:08:39.563998+0000","last_became_peered":"2023-11-28T11:08:39.563998+0000","last_unstale":"2023-11-28T11:08:39.570495+0000","last_undegraded":"2023-11-28T11:08:39.570495+0000","last_fullsized":"2023-11-28T11:08:39.570495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553351+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:08:39.553351+0000","last_peered":"2023-11-28T11:08:39.553351+0000","last_clean":"2023-11-28T11:08:39.553351+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:08:39.553351+0000","last_undegraded":"2023-11-28T11:08:39.553351+0000","last_fullsized":"2023-11-28T11:08:39.553351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546501+0000","last_change":"2023-11-28T11:08:35.141912+0000","last_active":"2023-11-28T11:08:39.546501+0000","last_peered":"2023-11-28T11:08:39.546501+0000","last_clean":"2023-11-28T11:08:39.546501+0000","last_became_active":"2023-11-28T11:08:35.141586+0000","last_became_peered":"2023-11-28T11:08:35.141586+0000","last_unstale":"2023-11-28T11:08:39.546501+0000","last_undegraded":"2023-11-28T11:08:39.546501+0000","last_fullsized":"2023-11-28T11:08:39.546501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552908+0000","last_change":"2023-11-28T11:08:35.542267+0000","last_active":"2023-11-28T11:08:39.552908+0000","last_peered":"2023-11-28T11:08:39.552908+0000","last_clean":"2023-11-28T11:08:39.552908+0000","last_became_active":"2023-11-28T11:08:35.541323+0000","last_became_peered":"2023-11-28T11:08:35.541323+0000","last_unstale":"2023-11-28T11:08:39.552908+0000","last_undegraded":"2023-11-28T11:08:39.552908+0000","last_fullsized":"2023-11-28T11:08:39.552908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570775+0000","last_change":"2023-11-28T11:08:39.566246+0000","last_active":"2023-11-28T11:08:39.570775+0000","last_peered":"2023-11-28T11:08:39.570775+0000","last_clean":"2023-11-28T11:08:39.570775+0000","last_became_active":"2023-11-28T11:08:39.565895+0000","last_became_peered":"2023-11-28T11:08:39.565895+0000","last_unstale":"2023-11-28T11:08:39.570775+0000","last_undegraded":"2023-11-28T11:08:39.570775+0000","last_fullsized":"2023-11-28T11:08:39.570775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556229+0000","last_change":"2023-11-28T11:08:35.146576+0000","last_active":"2023-11-28T11:08:39.556229+0000","last_peered":"2023-11-28T11:08:39.556229+0000","last_clean":"2023-11-28T11:08:39.556229+0000","last_became_active":"2023-11-28T11:08:35.146283+0000","last_became_peered":"2023-11-28T11:08:35.146283+0000","last_unstale":"2023-11-28T11:08:39.556229+0000","last_undegraded":"2023-11-28T11:08:39.556229+0000","last_fullsized":"2023-11-28T11:08:39.556229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542850+0000","last_change":"2023-11-28T11:08:35.536348+0000","last_active":"2023-11-28T11:08:39.542850+0000","last_peered":"2023-11-28T11:08:39.542850+0000","last_clean":"2023-11-28T11:08:39.542850+0000","last_became_active":"2023-11-28T11:08:35.536095+0000","last_became_peered":"2023-11-28T11:08:35.536095+0000","last_unstale":"2023-11-28T11:08:39.542850+0000","last_undegraded":"2023-11-28T11:08:39.542850+0000","last_fullsized":"2023-11-28T11:08:39.542850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.991268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570654+0000","last_change":"2023-11-28T11:08:39.567740+0000","last_active":"2023-11-28T11:08:39.570654+0000","last_peered":"2023-11-28T11:08:39.570654+0000","last_clean":"2023-11-28T11:08:39.570654+0000","last_became_active":"2023-11-28T11:08:39.567279+0000","last_became_peered":"2023-11-28T11:08:39.567279+0000","last_unstale":"2023-11-28T11:08:39.570654+0000","last_undegraded":"2023-11-28T11:08:39.570654+0000","last_fullsized":"2023-11-28T11:08:39.570654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546145+0000","last_change":"2023-11-28T11:08:35.141111+0000","last_active":"2023-11-28T11:08:39.546145+0000","last_peered":"2023-11-28T11:08:39.546145+0000","last_clean":"2023-11-28T11:08:39.546145+0000","last_became_active":"2023-11-28T11:08:35.140804+0000","last_became_peered":"2023-11-28T11:08:35.140804+0000","last_unstale":"2023-11-28T11:08:39.546145+0000","last_undegraded":"2023-11-28T11:08:39.546145+0000","last_fullsized":"2023-11-28T11:08:39.546145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556281+0000","last_change":"2023-11-28T11:08:35.547330+0000","last_active":"2023-11-28T11:08:39.556281+0000","last_peered":"2023-11-28T11:08:39.556281+0000","last_clean":"2023-11-28T11:08:39.556281+0000","last_became_active":"2023-11-28T11:08:35.546970+0000","last_became_peered":"2023-11-28T11:08:35.546970+0000","last_unstale":"2023-11-28T11:08:39.556281+0000","last_undegraded":"2023-11-28T11:08:39.556281+0000","last_fullsized":"2023-11-28T11:08:39.556281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:28.259978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568054+0000","last_change":"2023-11-28T11:08:39.565657+0000","last_active":"2023-11-28T11:08:39.568054+0000","last_peered":"2023-11-28T11:08:39.568054+0000","last_clean":"2023-11-28T11:08:39.568054+0000","last_became_active":"2023-11-28T11:08:39.565004+0000","last_became_peered":"2023-11-28T11:08:39.565004+0000","last_unstale":"2023-11-28T11:08:39.568054+0000","last_undegraded":"2023-11-28T11:08:39.568054+0000","last_fullsized":"2023-11-28T11:08:39.568054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546414+0000","last_change":"2023-11-28T11:08:35.146886+0000","last_active":"2023-11-28T11:08:39.546414+0000","last_peered":"2023-11-28T11:08:39.546414+0000","last_clean":"2023-11-28T11:08:39.546414+0000","last_became_active":"2023-11-28T11:08:35.146600+0000","last_became_peered":"2023-11-28T11:08:35.146600+0000","last_unstale":"2023-11-28T11:08:39.546414+0000","last_undegraded":"2023-11-28T11:08:39.546414+0000","last_fullsized":"2023-11-28T11:08:39.546414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556350+0000","last_change":"2023-11-28T11:08:35.538328+0000","last_active":"2023-11-28T11:08:39.556350+0000","last_peered":"2023-11-28T11:08:39.556350+0000","last_clean":"2023-11-28T11:08:39.556350+0000","last_became_active":"2023-11-28T11:08:35.537863+0000","last_became_peered":"2023-11-28T11:08:35.537863+0000","last_unstale":"2023-11-28T11:08:39.556350+0000","last_undegraded":"2023-11-28T11:08:39.556350+0000","last_fullsized":"2023-11-28T11:08:39.556350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566546+0000","last_change":"2023-11-28T11:08:39.562424+0000","last_active":"2023-11-28T11:08:39.566546+0000","last_peered":"2023-11-28T11:08:39.566546+0000","last_clean":"2023-11-28T11:08:39.566546+0000","last_became_active":"2023-11-28T11:08:39.562065+0000","last_became_peered":"2023-11-28T11:08:39.562065+0000","last_unstale":"2023-11-28T11:08:39.566546+0000","last_undegraded":"2023-11-28T11:08:39.566546+0000","last_fullsized":"2023-11-28T11:08:39.566546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:39.095254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544783+0000","last_change":"2023-11-28T11:08:35.138855+0000","last_active":"2023-11-28T11:08:39.544783+0000","last_peered":"2023-11-28T11:08:39.544783+0000","last_clean":"2023-11-28T11:08:39.544783+0000","last_became_active":"2023-11-28T11:08:35.138526+0000","last_became_peered":"2023-11-28T11:08:35.138526+0000","last_unstale":"2023-11-28T11:08:39.544783+0000","last_undegraded":"2023-11-28T11:08:39.544783+0000","last_fullsized":"2023-11-28T11:08:39.544783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544513+0000","last_change":"2023-11-28T11:08:35.539629+0000","last_active":"2023-11-28T11:08:39.544513+0000","last_peered":"2023-11-28T11:08:39.544513+0000","last_clean":"2023-11-28T11:08:39.544513+0000","last_became_active":"2023-11-28T11:08:35.534961+0000","last_became_peered":"2023-11-28T11:08:35.534961+0000","last_unstale":"2023-11-28T11:08:39.544513+0000","last_undegraded":"2023-11-28T11:08:39.544513+0000","last_fullsized":"2023-11-28T11:08:39.544513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571346+0000","last_change":"2023-11-28T11:08:39.570212+0000","last_active":"2023-11-28T11:08:39.571346+0000","last_peered":"2023-11-28T11:08:39.571346+0000","last_clean":"2023-11-28T11:08:39.571346+0000","last_became_active":"2023-11-28T11:08:39.569900+0000","last_became_peered":"2023-11-28T11:08:39.569900+0000","last_unstale":"2023-11-28T11:08:39.571346+0000","last_undegraded":"2023-11-28T11:08:39.571346+0000","last_fullsized":"2023-11-28T11:08:39.571346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552849+0000","last_change":"2023-11-28T11:08:35.139415+0000","last_active":"2023-11-28T11:08:39.552849+0000","last_peered":"2023-11-28T11:08:39.552849+0000","last_clean":"2023-11-28T11:08:39.552849+0000","last_became_active":"2023-11-28T11:08:35.139152+0000","last_became_peered":"2023-11-28T11:08:35.139152+0000","last_unstale":"2023-11-28T11:08:39.552849+0000","last_undegraded":"2023-11-28T11:08:39.552849+0000","last_fullsized":"2023-11-28T11:08:39.552849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552466+0000","last_change":"2023-11-28T11:08:35.544319+0000","last_active":"2023-11-28T11:08:39.552466+0000","last_peered":"2023-11-28T11:08:39.552466+0000","last_clean":"2023-11-28T11:08:39.552466+0000","last_became_active":"2023-11-28T11:08:35.544032+0000","last_became_peered":"2023-11-28T11:08:35.544032+0000","last_unstale":"2023-11-28T11:08:39.552466+0000","last_undegraded":"2023-11-28T11:08:39.552466+0000","last_fullsized":"2023-11-28T11:08:39.552466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:36.189248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551454+0000","last_change":"2023-11-28T11:08:35.534208+0000","last_active":"2023-11-28T11:08:39.551454+0000","last_peered":"2023-11-28T11:08:39.551454+0000","last_clean":"2023-11-28T11:08:39.551454+0000","last_became_active":"2023-11-28T11:08:35.533929+0000","last_became_peered":"2023-11-28T11:08:35.533929+0000","last_unstale":"2023-11-28T11:08:39.551454+0000","last_undegraded":"2023-11-28T11:08:39.551454+0000","last_fullsized":"2023-11-28T11:08:39.551454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:44.139630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544658+0000","last_change":"2023-11-28T11:08:35.540664+0000","last_active":"2023-11-28T11:08:39.544658+0000","last_peered":"2023-11-28T11:08:39.544658+0000","last_clean":"2023-11-28T11:08:39.544658+0000","last_became_active":"2023-11-28T11:08:35.540243+0000","last_became_peered":"2023-11-28T11:08:35.540243+0000","last_unstale":"2023-11-28T11:08:39.544658+0000","last_undegraded":"2023-11-28T11:08:39.544658+0000","last_fullsized":"2023-11-28T11:08:39.544658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556410+0000","last_change":"2023-11-28T11:08:35.544372+0000","last_active":"2023-11-28T11:08:39.556410+0000","last_peered":"2023-11-28T11:08:39.556410+0000","last_clean":"2023-11-28T11:08:39.556410+0000","last_became_active":"2023-11-28T11:08:35.543867+0000","last_became_peered":"2023-11-28T11:08:35.543867+0000","last_unstale":"2023-11-28T11:08:39.556410+0000","last_undegraded":"2023-11-28T11:08:39.556410+0000","last_fullsized":"2023-11-28T11:08:39.556410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:30.327766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544832+0000","last_change":"2023-11-28T11:08:35.543075+0000","last_active":"2023-11-28T11:08:39.544832+0000","last_peered":"2023-11-28T11:08:39.544832+0000","last_clean":"2023-11-28T11:08:39.544832+0000","last_became_active":"2023-11-28T11:08:35.542645+0000","last_became_peered":"2023-11-28T11:08:35.542645+0000","last_unstale":"2023-11-28T11:08:39.544832+0000","last_undegraded":"2023-11-28T11:08:39.544832+0000","last_fullsized":"2023-11-28T11:08:39.544832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555411+0000","last_change":"2023-11-28T11:08:35.546225+0000","last_active":"2023-11-28T11:08:39.555411+0000","last_peered":"2023-11-28T11:08:39.555411+0000","last_clean":"2023-11-28T11:08:39.555411+0000","last_became_active":"2023-11-28T11:08:35.545970+0000","last_became_peered":"2023-11-28T11:08:35.545970+0000","last_unstale":"2023-11-28T11:08:39.555411+0000","last_undegraded":"2023-11-28T11:08:39.555411+0000","last_fullsized":"2023-11-28T11:08:39.555411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4387,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9248,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362592,"statfs":{"total":96636764160,"available":96627294208,"internally_reserved":0,"allocated":876544,"data_stored":698919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8060,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363780,"statfs":{"total":96636764160,"available":96628510720,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8128,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363712,"statfs":{"total":96636764160,"available":96628441088,"internally_reserved":0,"allocated":253952,"data_stored":103935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8064,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363776,"statfs":{"total":96636764160,"available":96628506624,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8132,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363708,"statfs":{"total":96636764160,"available":96628436992,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8212,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363628,"statfs":{"total":96636764160,"available":96628355072,"internally_reserved":0,"allocated":274432,"data_stored":108391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":105366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:52.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:53.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.091+0000 7f4b53f1e700 1 Processor -- start 2023-11-28T11:08:53.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.091+0000 7f4b53f1e700 1 -- start start 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b53f1e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b4c15cd10 0x7f4b4c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b53f1e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4b4c15d0e0 0x7f4b4c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b53f1e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4c0cbfb0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b53f1e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c06cee0 con 0x7f4b4c15cd10 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b53f1e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c0cb710 con 0x7f4b4c15d0e0 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b51cba700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b4c15cd10 0x7f4b4c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b524bb700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4b4c0b18a0 0x7f4b4c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46554/0 (socket says 172.21.15.83:46554) 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b51cba700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b4c15cd10 0x7f4b4c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48690/0 (socket says 172.21.15.83:48690) 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.092+0000 7f4b524bb700 1 -- 172.21.15.83:0/683785514 learned_addr learned my addr 172.21.15.83:0/683785514 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b514b9700 1 --2- 172.21.15.83:0/683785514 >> v2:172.21.15.90:3300/0 conn(0x7f4b4c15d0e0 0x7f4b4c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b51cba700 1 -- 172.21.15.83:0/683785514 >> v2:172.21.15.90:3300/0 conn(0x7f4b4c15d0e0 msgr2=0x7f4b4c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b51cba700 1 --2- 172.21.15.83:0/683785514 >> v2:172.21.15.90:3300/0 conn(0x7f4b4c15d0e0 0x7f4b4c0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b51cba700 1 -- 172.21.15.83:0/683785514 >> v1:172.21.15.83:6789/0 conn(0x7f4b4c0b18a0 legacy=0x7f4b4c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b51cba700 1 -- 172.21.15.83:0/683785514 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c0cb4e0 con 0x7f4b4c15cd10 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b50cb8700 1 -- 172.21.15.83:0/683785514 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456646539 0 0) 0x7f4b4c0cbfb0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b51cba700 1 --2- 172.21.15.83:0/683785514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0ca970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0cc800 tx=0x7f4b40009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5893b18f309046 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b50cb8700 1 -- 172.21.15.83:0/683785514 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40011830 con 0x7f4b4c15cd10 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b50cb8700 1 -- 172.21.15.83:0/683785514 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b40011990 con 0x7f4b4c15cd10 2023-11-28T11:08:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.093+0000 7f4b50cb8700 1 -- 172.21.15.83:0/683785514 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40011d20 con 0x7f4b4c15cd10 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 -- 172.21.15.83:0/683785514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c15cd10 msgr2=0x7f4b4c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/683785514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0ca970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0cc800 tx=0x7f4b40009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 -- 172.21.15.83:0/683785514 shutdown_connections 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/683785514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0ca970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/683785514 >> v2:172.21.15.90:3300/0 conn(0x7f4b4c15d0e0 0x7f4b4c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 -- 172.21.15.83:0/683785514 >> 172.21.15.83:0/683785514 conn(0x7f4b4c06e9a0 msgr2=0x7f4b4c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 -- 172.21.15.83:0/683785514 shutdown_connections 2023-11-28T11:08:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.094+0000 7f4b53f1e700 1 -- 172.21.15.83:0/683785514 wait complete. 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.095+0000 7f4b53f1e700 1 Processor -- start 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.095+0000 7f4b53f1e700 1 -- start start 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b53f1e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 0x7f4b4c15e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b53f1e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b53f1e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4c15d0e0 0x7f4b4c0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b53f1e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c15ff80 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b53f1e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c15e6f0 con 0x7f4b4c15cd10 2023-11-28T11:08:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b53f1e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c159ac0 con 0x7f4b4c15d0e0 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b51cba700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 0x7f4b4c15e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b51cba700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 0x7f4b4c15e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48696/0 (socket says 172.21.15.83:48696) 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b51cba700 1 -- 172.21.15.83:0/2757769670 learned_addr learned my addr 172.21.15.83:0/2757769670 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b514b9700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.096+0000 7f4b524bb700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4c15d0e0 0x7f4b4c0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b51cba700 1 -- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4c15d0e0 msgr2=0x7f4b4c0b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b51cba700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4c15d0e0 0x7f4b4c0b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b51cba700 1 -- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4c15cd10 msgr2=0x7f4b4c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b51cba700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b51cba700 1 -- 172.21.15.83:0/2757769670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400094a0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b524bb700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4c15d0e0 0x7f4b4c0b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b51cba700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 0x7f4b4c15e1b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b400090b0 tx=0x7f4b40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b400036a0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4001f070 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c0cbcd0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40003a00 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.097+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c0cb9f0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.098+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4c06b9b0 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.098+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b4000f050 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.099+0000 7f4b3e7fc700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4b300682f0 0x7f4b3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.099+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4b40021070 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.099+0000 7f4b514b9700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4b300682f0 0x7f4b3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.099+0000 7f4b514b9700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4b300682f0 0x7f4b3006a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c155f30 tx=0x7f4b48008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.103+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b40053e40 con 0x7f4b4c0b18a0 2023-11-28T11:08:53.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.279+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b4c074320 con 0x7f4b300682f0 2023-11-28T11:08:53.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.288+0000 7f4b3e7fc700 1 -- 172.21.15.83:0/2757769670 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367582 (secure 0 0 0) 0x7f4b4c074320 con 0x7f4b300682f0 2023-11-28T11:08:53.294 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4b300682f0 msgr2=0x7f4b3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4b300682f0 0x7f4b3006a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c155f30 tx=0x7f4b48008040 comp rx=0 tx=0).stop 2023-11-28T11:08:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 msgr2=0x7f4b4c15e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 0x7f4b4c15e1b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b400090b0 tx=0x7f4b40004750 comp rx=0 tx=0).stop 2023-11-28T11:08:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 shutdown_connections 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4c15d0e0 0x7f4b4c0b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4b300682f0 0x7f4b3006a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b4c0b18a0 0x7f4b4c15e1b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 --2- 172.21.15.83:0/2757769670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4c15cd10 0x7f4b4c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 >> 172.21.15.83:0/2757769670 conn(0x7f4b4c06e9a0 msgr2=0x7f4b4c15c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 shutdown_connections 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.292+0000 7f4b53f1e700 1 -- 172.21.15.83:0/2757769670 wait complete. 2023-11-28T11:08:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:08:53.305 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-11-28T11:08:51.572164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594755,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67136,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907584,"statfs":{"total":773094113280,"available":773025366016,"internally_reserved":0,"allocated":3276800,"data_stored":2024628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2057,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.625161"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542681+0000","last_change":"2023-11-28T11:08:35.535687+0000","last_active":"2023-11-28T11:08:39.542681+0000","last_peered":"2023-11-28T11:08:39.542681+0000","last_clean":"2023-11-28T11:08:39.542681+0000","last_became_active":"2023-11-28T11:08:35.535025+0000","last_became_peered":"2023-11-28T11:08:35.535025+0000","last_unstale":"2023-11-28T11:08:39.542681+0000","last_undegraded":"2023-11-28T11:08:39.542681+0000","last_fullsized":"2023-11-28T11:08:39.542681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:21.578670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546994+0000","last_change":"2023-11-28T11:08:35.534288+0000","last_active":"2023-11-28T11:08:39.546994+0000","last_peered":"2023-11-28T11:08:39.546994+0000","last_clean":"2023-11-28T11:08:39.546994+0000","last_became_active":"2023-11-28T11:08:35.533997+0000","last_became_peered":"2023-11-28T11:08:35.533997+0000","last_unstale":"2023-11-28T11:08:39.546994+0000","last_undegraded":"2023-11-28T11:08:39.546994+0000","last_fullsized":"2023-11-28T11:08:39.546994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556945+0000","last_change":"2023-11-28T11:08:35.553042+0000","last_active":"2023-11-28T11:08:39.556945+0000","last_peered":"2023-11-28T11:08:39.556945+0000","last_clean":"2023-11-28T11:08:39.556945+0000","last_became_active":"2023-11-28T11:08:35.552693+0000","last_became_peered":"2023-11-28T11:08:35.552693+0000","last_unstale":"2023-11-28T11:08:39.556945+0000","last_undegraded":"2023-11-28T11:08:39.556945+0000","last_fullsized":"2023-11-28T11:08:39.556945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:43.204644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551967+0000","last_change":"2023-11-28T11:08:35.538585+0000","last_active":"2023-11-28T11:08:39.551967+0000","last_peered":"2023-11-28T11:08:39.551967+0000","last_clean":"2023-11-28T11:08:39.551967+0000","last_became_active":"2023-11-28T11:08:35.538074+0000","last_became_peered":"2023-11-28T11:08:35.538074+0000","last_unstale":"2023-11-28T11:08:39.551967+0000","last_undegraded":"2023-11-28T11:08:39.551967+0000","last_fullsized":"2023-11-28T11:08:39.551967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:13.796245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556389+0000","last_change":"2023-11-28T11:08:35.548649+0000","last_active":"2023-11-28T11:08:39.556389+0000","last_peered":"2023-11-28T11:08:39.556389+0000","last_clean":"2023-11-28T11:08:39.556389+0000","last_became_active":"2023-11-28T11:08:35.546646+0000","last_became_peered":"2023-11-28T11:08:35.546646+0000","last_unstale":"2023-11-28T11:08:39.556389+0000","last_undegraded":"2023-11-28T11:08:39.556389+0000","last_fullsized":"2023-11-28T11:08:39.556389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556449+0000","last_change":"2023-11-28T11:08:35.545320+0000","last_active":"2023-11-28T11:08:39.556449+0000","last_peered":"2023-11-28T11:08:39.556449+0000","last_clean":"2023-11-28T11:08:39.556449+0000","last_became_active":"2023-11-28T11:08:35.545051+0000","last_became_peered":"2023-11-28T11:08:35.545051+0000","last_unstale":"2023-11-28T11:08:39.556449+0000","last_undegraded":"2023-11-28T11:08:39.556449+0000","last_fullsized":"2023-11-28T11:08:39.556449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549589+0000","last_change":"2023-11-28T11:08:35.543641+0000","last_active":"2023-11-28T11:08:39.549589+0000","last_peered":"2023-11-28T11:08:39.549589+0000","last_clean":"2023-11-28T11:08:39.549589+0000","last_became_active":"2023-11-28T11:08:35.543407+0000","last_became_peered":"2023-11-28T11:08:35.543407+0000","last_unstale":"2023-11-28T11:08:39.549589+0000","last_undegraded":"2023-11-28T11:08:39.549589+0000","last_fullsized":"2023-11-28T11:08:39.549589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546097+0000","last_change":"2023-11-28T11:08:35.538922+0000","last_active":"2023-11-28T11:08:39.546097+0000","last_peered":"2023-11-28T11:08:39.546097+0000","last_clean":"2023-11-28T11:08:39.546097+0000","last_became_active":"2023-11-28T11:08:35.538455+0000","last_became_peered":"2023-11-28T11:08:35.538455+0000","last_unstale":"2023-11-28T11:08:39.546097+0000","last_undegraded":"2023-11-28T11:08:39.546097+0000","last_fullsized":"2023-11-28T11:08:39.546097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549757+0000","last_change":"2023-11-28T11:08:35.550345+0000","last_active":"2023-11-28T11:08:39.549757+0000","last_peered":"2023-11-28T11:08:39.549757+0000","last_clean":"2023-11-28T11:08:39.549757+0000","last_became_active":"2023-11-28T11:08:35.546222+0000","last_became_peered":"2023-11-28T11:08:35.546222+0000","last_unstale":"2023-11-28T11:08:39.549757+0000","last_undegraded":"2023-11-28T11:08:39.549757+0000","last_fullsized":"2023-11-28T11:08:39.549757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546977+0000","last_change":"2023-11-28T11:08:35.550152+0000","last_active":"2023-11-28T11:08:39.546977+0000","last_peered":"2023-11-28T11:08:39.546977+0000","last_clean":"2023-11-28T11:08:39.546977+0000","last_became_active":"2023-11-28T11:08:35.549618+0000","last_became_peered":"2023-11-28T11:08:35.549618+0000","last_unstale":"2023-11-28T11:08:39.546977+0000","last_undegraded":"2023-11-28T11:08:39.546977+0000","last_fullsized":"2023-11-28T11:08:39.546977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556901+0000","last_change":"2023-11-28T11:08:35.544292+0000","last_active":"2023-11-28T11:08:39.556901+0000","last_peered":"2023-11-28T11:08:39.556901+0000","last_clean":"2023-11-28T11:08:39.556901+0000","last_became_active":"2023-11-28T11:08:35.543905+0000","last_became_peered":"2023-11-28T11:08:35.543905+0000","last_unstale":"2023-11-28T11:08:39.556901+0000","last_undegraded":"2023-11-28T11:08:39.556901+0000","last_fullsized":"2023-11-28T11:08:39.556901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:46.079543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542701+0000","last_change":"2023-11-28T11:08:35.548959+0000","last_active":"2023-11-28T11:08:39.542701+0000","last_peered":"2023-11-28T11:08:39.542701+0000","last_clean":"2023-11-28T11:08:39.542701+0000","last_became_active":"2023-11-28T11:08:35.548707+0000","last_became_peered":"2023-11-28T11:08:35.548707+0000","last_unstale":"2023-11-28T11:08:39.542701+0000","last_undegraded":"2023-11-28T11:08:39.542701+0000","last_fullsized":"2023-11-28T11:08:39.542701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:41.426965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542575+0000","last_change":"2023-11-28T11:08:35.534783+0000","last_active":"2023-11-28T11:08:39.542575+0000","last_peered":"2023-11-28T11:08:39.542575+0000","last_clean":"2023-11-28T11:08:39.542575+0000","last_became_active":"2023-11-28T11:08:35.534520+0000","last_became_peered":"2023-11-28T11:08:35.534520+0000","last_unstale":"2023-11-28T11:08:39.542575+0000","last_undegraded":"2023-11-28T11:08:39.542575+0000","last_fullsized":"2023-11-28T11:08:39.542575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:56.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546312+0000","last_change":"2023-11-28T11:08:35.538322+0000","last_active":"2023-11-28T11:08:39.546312+0000","last_peered":"2023-11-28T11:08:39.546312+0000","last_clean":"2023-11-28T11:08:39.546312+0000","last_became_active":"2023-11-28T11:08:35.538000+0000","last_became_peered":"2023-11-28T11:08:35.538000+0000","last_unstale":"2023-11-28T11:08:39.546312+0000","last_undegraded":"2023-11-28T11:08:39.546312+0000","last_fullsized":"2023-11-28T11:08:39.546312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549399+0000","last_change":"2023-11-28T11:08:35.544771+0000","last_active":"2023-11-28T11:08:39.549399+0000","last_peered":"2023-11-28T11:08:39.549399+0000","last_clean":"2023-11-28T11:08:39.549399+0000","last_became_active":"2023-11-28T11:08:35.544525+0000","last_became_peered":"2023-11-28T11:08:35.544525+0000","last_unstale":"2023-11-28T11:08:39.549399+0000","last_undegraded":"2023-11-28T11:08:39.549399+0000","last_fullsized":"2023-11-28T11:08:39.549399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552566+0000","last_change":"2023-11-28T11:08:35.547686+0000","last_active":"2023-11-28T11:08:39.552566+0000","last_peered":"2023-11-28T11:08:39.552566+0000","last_clean":"2023-11-28T11:08:39.552566+0000","last_became_active":"2023-11-28T11:08:35.547411+0000","last_became_peered":"2023-11-28T11:08:35.547411+0000","last_unstale":"2023-11-28T11:08:39.552566+0000","last_undegraded":"2023-11-28T11:08:39.552566+0000","last_fullsized":"2023-11-28T11:08:39.552566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549830+0000","last_change":"2023-11-28T11:08:35.551541+0000","last_active":"2023-11-28T11:08:39.549830+0000","last_peered":"2023-11-28T11:08:39.549830+0000","last_clean":"2023-11-28T11:08:39.549830+0000","last_became_active":"2023-11-28T11:08:35.546691+0000","last_became_peered":"2023-11-28T11:08:35.546691+0000","last_unstale":"2023-11-28T11:08:39.549830+0000","last_undegraded":"2023-11-28T11:08:39.549830+0000","last_fullsized":"2023-11-28T11:08:39.549830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556352+0000","last_change":"2023-11-28T11:08:35.541127+0000","last_active":"2023-11-28T11:08:39.556352+0000","last_peered":"2023-11-28T11:08:39.556352+0000","last_clean":"2023-11-28T11:08:39.556352+0000","last_became_active":"2023-11-28T11:08:35.540800+0000","last_became_peered":"2023-11-28T11:08:35.540800+0000","last_unstale":"2023-11-28T11:08:39.556352+0000","last_undegraded":"2023-11-28T11:08:39.556352+0000","last_fullsized":"2023-11-28T11:08:39.556352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.543580+0000","last_change":"2023-11-28T11:08:35.546695+0000","last_active":"2023-11-28T11:08:39.543580+0000","last_peered":"2023-11-28T11:08:39.543580+0000","last_clean":"2023-11-28T11:08:39.543580+0000","last_became_active":"2023-11-28T11:08:35.546438+0000","last_became_peered":"2023-11-28T11:08:35.546438+0000","last_unstale":"2023-11-28T11:08:39.543580+0000","last_undegraded":"2023-11-28T11:08:39.543580+0000","last_fullsized":"2023-11-28T11:08:39.543580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:37.007416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556699+0000","last_change":"2023-11-28T11:08:35.548019+0000","last_active":"2023-11-28T11:08:39.556699+0000","last_peered":"2023-11-28T11:08:39.556699+0000","last_clean":"2023-11-28T11:08:39.556699+0000","last_became_active":"2023-11-28T11:08:35.541157+0000","last_became_peered":"2023-11-28T11:08:35.541157+0000","last_unstale":"2023-11-28T11:08:39.556699+0000","last_undegraded":"2023-11-28T11:08:39.556699+0000","last_fullsized":"2023-11-28T11:08:39.556699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:40.346506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551701+0000","last_change":"2023-11-28T11:08:35.540145+0000","last_active":"2023-11-28T11:08:39.551701+0000","last_peered":"2023-11-28T11:08:39.551701+0000","last_clean":"2023-11-28T11:08:39.551701+0000","last_became_active":"2023-11-28T11:08:35.539880+0000","last_became_peered":"2023-11-28T11:08:35.539880+0000","last_unstale":"2023-11-28T11:08:39.551701+0000","last_undegraded":"2023-11-28T11:08:39.551701+0000","last_fullsized":"2023-11-28T11:08:39.551701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549283+0000","last_change":"2023-11-28T11:08:35.546609+0000","last_active":"2023-11-28T11:08:39.549283+0000","last_peered":"2023-11-28T11:08:39.549283+0000","last_clean":"2023-11-28T11:08:39.549283+0000","last_became_active":"2023-11-28T11:08:35.545637+0000","last_became_peered":"2023-11-28T11:08:35.545637+0000","last_unstale":"2023-11-28T11:08:39.549283+0000","last_undegraded":"2023-11-28T11:08:39.549283+0000","last_fullsized":"2023-11-28T11:08:39.549283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544746+0000","last_change":"2023-11-28T11:08:35.540494+0000","last_active":"2023-11-28T11:08:39.544746+0000","last_peered":"2023-11-28T11:08:39.544746+0000","last_clean":"2023-11-28T11:08:39.544746+0000","last_became_active":"2023-11-28T11:08:35.539936+0000","last_became_peered":"2023-11-28T11:08:35.539936+0000","last_unstale":"2023-11-28T11:08:39.544746+0000","last_undegraded":"2023-11-28T11:08:39.544746+0000","last_fullsized":"2023-11-28T11:08:39.544746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550772+0000","last_change":"2023-11-28T11:08:35.545757+0000","last_active":"2023-11-28T11:08:39.550772+0000","last_peered":"2023-11-28T11:08:39.550772+0000","last_clean":"2023-11-28T11:08:39.550772+0000","last_became_active":"2023-11-28T11:08:35.545459+0000","last_became_peered":"2023-11-28T11:08:35.545459+0000","last_unstale":"2023-11-28T11:08:39.550772+0000","last_undegraded":"2023-11-28T11:08:39.550772+0000","last_fullsized":"2023-11-28T11:08:39.550772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556095+0000","last_change":"2023-11-28T11:08:35.540552+0000","last_active":"2023-11-28T11:08:39.556095+0000","last_peered":"2023-11-28T11:08:39.556095+0000","last_clean":"2023-11-28T11:08:39.556095+0000","last_became_active":"2023-11-28T11:08:35.539972+0000","last_became_peered":"2023-11-28T11:08:35.539972+0000","last_unstale":"2023-11-28T11:08:39.556095+0000","last_undegraded":"2023-11-28T11:08:39.556095+0000","last_fullsized":"2023-11-28T11:08:39.556095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556230+0000","last_change":"2023-11-28T11:08:35.536499+0000","last_active":"2023-11-28T11:08:39.556230+0000","last_peered":"2023-11-28T11:08:39.556230+0000","last_clean":"2023-11-28T11:08:39.556230+0000","last_became_active":"2023-11-28T11:08:35.536308+0000","last_became_peered":"2023-11-28T11:08:35.536308+0000","last_unstale":"2023-11-28T11:08:39.556230+0000","last_undegraded":"2023-11-28T11:08:39.556230+0000","last_fullsized":"2023-11-28T11:08:39.556230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546678+0000","last_change":"2023-11-28T11:08:35.542161+0000","last_active":"2023-11-28T11:08:39.546678+0000","last_peered":"2023-11-28T11:08:39.546678+0000","last_clean":"2023-11-28T11:08:39.546678+0000","last_became_active":"2023-11-28T11:08:35.541331+0000","last_became_peered":"2023-11-28T11:08:35.541331+0000","last_unstale":"2023-11-28T11:08:39.546678+0000","last_undegraded":"2023-11-28T11:08:39.546678+0000","last_fullsized":"2023-11-28T11:08:39.546678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570777+0000","last_change":"2023-11-28T11:08:39.566823+0000","last_active":"2023-11-28T11:08:39.570777+0000","last_peered":"2023-11-28T11:08:39.570777+0000","last_clean":"2023-11-28T11:08:39.570777+0000","last_became_active":"2023-11-28T11:08:39.563462+0000","last_became_peered":"2023-11-28T11:08:39.563462+0000","last_unstale":"2023-11-28T11:08:39.570777+0000","last_undegraded":"2023-11-28T11:08:39.570777+0000","last_fullsized":"2023-11-28T11:08:39.570777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:25.821485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556318+0000","last_change":"2023-11-28T11:08:35.144920+0000","last_active":"2023-11-28T11:08:39.556318+0000","last_peered":"2023-11-28T11:08:39.556318+0000","last_clean":"2023-11-28T11:08:39.556318+0000","last_became_active":"2023-11-28T11:08:35.144360+0000","last_became_peered":"2023-11-28T11:08:35.144360+0000","last_unstale":"2023-11-28T11:08:39.556318+0000","last_undegraded":"2023-11-28T11:08:39.556318+0000","last_fullsized":"2023-11-28T11:08:39.556318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544509+0000","last_change":"2023-11-28T11:08:35.537580+0000","last_active":"2023-11-28T11:08:39.544509+0000","last_peered":"2023-11-28T11:08:39.544509+0000","last_clean":"2023-11-28T11:08:39.544509+0000","last_became_active":"2023-11-28T11:08:35.537300+0000","last_became_peered":"2023-11-28T11:08:35.537300+0000","last_unstale":"2023-11-28T11:08:39.544509+0000","last_undegraded":"2023-11-28T11:08:39.544509+0000","last_fullsized":"2023-11-28T11:08:39.544509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573888+0000","last_change":"2023-11-28T11:08:39.565761+0000","last_active":"2023-11-28T11:08:39.573888+0000","last_peered":"2023-11-28T11:08:39.573888+0000","last_clean":"2023-11-28T11:08:39.573888+0000","last_became_active":"2023-11-28T11:08:39.565387+0000","last_became_peered":"2023-11-28T11:08:39.565387+0000","last_unstale":"2023-11-28T11:08:39.573888+0000","last_undegraded":"2023-11-28T11:08:39.573888+0000","last_fullsized":"2023-11-28T11:08:39.573888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:29.648074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552645+0000","last_change":"2023-11-28T11:08:35.141536+0000","last_active":"2023-11-28T11:08:39.552645+0000","last_peered":"2023-11-28T11:08:39.552645+0000","last_clean":"2023-11-28T11:08:39.552645+0000","last_became_active":"2023-11-28T11:08:35.141259+0000","last_became_peered":"2023-11-28T11:08:35.141259+0000","last_unstale":"2023-11-28T11:08:39.552645+0000","last_undegraded":"2023-11-28T11:08:39.552645+0000","last_fullsized":"2023-11-28T11:08:39.552645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551925+0000","last_change":"2023-11-28T11:08:35.551235+0000","last_active":"2023-11-28T11:08:39.551925+0000","last_peered":"2023-11-28T11:08:39.551925+0000","last_clean":"2023-11-28T11:08:39.551925+0000","last_became_active":"2023-11-28T11:08:35.550945+0000","last_became_peered":"2023-11-28T11:08:35.550945+0000","last_unstale":"2023-11-28T11:08:39.551925+0000","last_undegraded":"2023-11-28T11:08:39.551925+0000","last_fullsized":"2023-11-28T11:08:39.551925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.576247+0000","last_change":"2023-11-28T11:08:39.574297+0000","last_active":"2023-11-28T11:08:39.576247+0000","last_peered":"2023-11-28T11:08:39.576247+0000","last_clean":"2023-11-28T11:08:39.576247+0000","last_became_active":"2023-11-28T11:08:39.573836+0000","last_became_peered":"2023-11-28T11:08:39.573836+0000","last_unstale":"2023-11-28T11:08:39.576247+0000","last_undegraded":"2023-11-28T11:08:39.576247+0000","last_fullsized":"2023-11-28T11:08:39.576247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546095+0000","last_change":"2023-11-28T11:08:35.140807+0000","last_active":"2023-11-28T11:08:39.546095+0000","last_peered":"2023-11-28T11:08:39.546095+0000","last_clean":"2023-11-28T11:08:39.546095+0000","last_became_active":"2023-11-28T11:08:35.140551+0000","last_became_peered":"2023-11-28T11:08:35.140551+0000","last_unstale":"2023-11-28T11:08:39.546095+0000","last_undegraded":"2023-11-28T11:08:39.546095+0000","last_fullsized":"2023-11-28T11:08:39.546095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553480+0000","last_change":"2023-11-28T11:08:35.540497+0000","last_active":"2023-11-28T11:08:39.553480+0000","last_peered":"2023-11-28T11:08:39.553480+0000","last_clean":"2023-11-28T11:08:39.553480+0000","last_became_active":"2023-11-28T11:08:35.540154+0000","last_became_peered":"2023-11-28T11:08:35.540154+0000","last_unstale":"2023-11-28T11:08:39.553480+0000","last_undegraded":"2023-11-28T11:08:39.553480+0000","last_fullsized":"2023-11-28T11:08:39.553480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:43.354051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.574001+0000","last_change":"2023-11-28T11:08:39.568636+0000","last_active":"2023-11-28T11:08:39.574001+0000","last_peered":"2023-11-28T11:08:39.574001+0000","last_clean":"2023-11-28T11:08:39.574001+0000","last_became_active":"2023-11-28T11:08:39.568255+0000","last_became_peered":"2023-11-28T11:08:39.568255+0000","last_unstale":"2023-11-28T11:08:39.574001+0000","last_undegraded":"2023-11-28T11:08:39.574001+0000","last_fullsized":"2023-11-28T11:08:39.574001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549903+0000","last_change":"2023-11-28T11:08:35.139702+0000","last_active":"2023-11-28T11:08:39.549903+0000","last_peered":"2023-11-28T11:08:39.549903+0000","last_clean":"2023-11-28T11:08:39.549903+0000","last_became_active":"2023-11-28T11:08:35.139379+0000","last_became_peered":"2023-11-28T11:08:35.139379+0000","last_unstale":"2023-11-28T11:08:39.549903+0000","last_undegraded":"2023-11-28T11:08:39.549903+0000","last_fullsized":"2023-11-28T11:08:39.549903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546007+0000","last_change":"2023-11-28T11:08:35.532733+0000","last_active":"2023-11-28T11:08:39.546007+0000","last_peered":"2023-11-28T11:08:39.546007+0000","last_clean":"2023-11-28T11:08:39.546007+0000","last_became_active":"2023-11-28T11:08:35.532390+0000","last_became_peered":"2023-11-28T11:08:35.532390+0000","last_unstale":"2023-11-28T11:08:39.546007+0000","last_undegraded":"2023-11-28T11:08:39.546007+0000","last_fullsized":"2023-11-28T11:08:39.546007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573440+0000","last_change":"2023-11-28T11:08:39.568834+0000","last_active":"2023-11-28T11:08:39.573440+0000","last_peered":"2023-11-28T11:08:39.573440+0000","last_clean":"2023-11-28T11:08:39.573440+0000","last_became_active":"2023-11-28T11:08:39.568153+0000","last_became_peered":"2023-11-28T11:08:39.568153+0000","last_unstale":"2023-11-28T11:08:39.573440+0000","last_undegraded":"2023-11-28T11:08:39.573440+0000","last_fullsized":"2023-11-28T11:08:39.573440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546860+0000","last_change":"2023-11-28T11:08:35.140772+0000","last_active":"2023-11-28T11:08:39.546860+0000","last_peered":"2023-11-28T11:08:39.546860+0000","last_clean":"2023-11-28T11:08:39.546860+0000","last_became_active":"2023-11-28T11:08:35.140418+0000","last_became_peered":"2023-11-28T11:08:35.140418+0000","last_unstale":"2023-11-28T11:08:39.546860+0000","last_undegraded":"2023-11-28T11:08:39.546860+0000","last_fullsized":"2023-11-28T11:08:39.546860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544363+0000","last_change":"2023-11-28T11:08:35.544651+0000","last_active":"2023-11-28T11:08:39.544363+0000","last_peered":"2023-11-28T11:08:39.544363+0000","last_clean":"2023-11-28T11:08:39.544363+0000","last_became_active":"2023-11-28T11:08:35.541280+0000","last_became_peered":"2023-11-28T11:08:35.541280+0000","last_unstale":"2023-11-28T11:08:39.544363+0000","last_undegraded":"2023-11-28T11:08:39.544363+0000","last_fullsized":"2023-11-28T11:08:39.544363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568222+0000","last_change":"2023-11-28T11:08:39.562858+0000","last_active":"2023-11-28T11:08:39.568222+0000","last_peered":"2023-11-28T11:08:39.568222+0000","last_clean":"2023-11-28T11:08:39.568222+0000","last_became_active":"2023-11-28T11:08:39.562273+0000","last_became_peered":"2023-11-28T11:08:39.562273+0000","last_unstale":"2023-11-28T11:08:39.568222+0000","last_undegraded":"2023-11-28T11:08:39.568222+0000","last_fullsized":"2023-11-28T11:08:39.568222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:39.831045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552124+0000","last_change":"2023-11-28T11:08:35.144557+0000","last_active":"2023-11-28T11:08:39.552124+0000","last_peered":"2023-11-28T11:08:39.552124+0000","last_clean":"2023-11-28T11:08:39.552124+0000","last_became_active":"2023-11-28T11:08:35.144222+0000","last_became_peered":"2023-11-28T11:08:35.144222+0000","last_unstale":"2023-11-28T11:08:39.552124+0000","last_undegraded":"2023-11-28T11:08:39.552124+0000","last_fullsized":"2023-11-28T11:08:39.552124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552994+0000","last_change":"2023-11-28T11:08:35.543394+0000","last_active":"2023-11-28T11:08:39.552994+0000","last_peered":"2023-11-28T11:08:39.552994+0000","last_clean":"2023-11-28T11:08:39.552994+0000","last_became_active":"2023-11-28T11:08:35.543099+0000","last_became_peered":"2023-11-28T11:08:35.543099+0000","last_unstale":"2023-11-28T11:08:39.552994+0000","last_undegraded":"2023-11-28T11:08:39.552994+0000","last_fullsized":"2023-11-28T11:08:39.552994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571358+0000","last_change":"2023-11-28T11:08:39.570301+0000","last_active":"2023-11-28T11:08:39.571358+0000","last_peered":"2023-11-28T11:08:39.571358+0000","last_clean":"2023-11-28T11:08:39.571358+0000","last_became_active":"2023-11-28T11:08:39.569828+0000","last_became_peered":"2023-11-28T11:08:39.569828+0000","last_unstale":"2023-11-28T11:08:39.571358+0000","last_undegraded":"2023-11-28T11:08:39.571358+0000","last_fullsized":"2023-11-28T11:08:39.571358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:43.778416+0000","last_change":"2023-11-28T11:08:35.147185+0000","last_active":"2023-11-28T11:08:43.778416+0000","last_peered":"2023-11-28T11:08:43.778416+0000","last_clean":"2023-11-28T11:08:43.778416+0000","last_became_active":"2023-11-28T11:08:35.146285+0000","last_became_peered":"2023-11-28T11:08:35.146285+0000","last_unstale":"2023-11-28T11:08:43.778416+0000","last_undegraded":"2023-11-28T11:08:43.778416+0000","last_fullsized":"2023-11-28T11:08:43.778416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3605,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556565+0000","last_change":"2023-11-28T11:08:35.546895+0000","last_active":"2023-11-28T11:08:39.556565+0000","last_peered":"2023-11-28T11:08:39.556565+0000","last_clean":"2023-11-28T11:08:39.556565+0000","last_became_active":"2023-11-28T11:08:35.538447+0000","last_became_peered":"2023-11-28T11:08:35.538447+0000","last_unstale":"2023-11-28T11:08:39.556565+0000","last_undegraded":"2023-11-28T11:08:39.556565+0000","last_fullsized":"2023-11-28T11:08:39.556565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:28.436759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564343+0000","last_change":"2023-11-28T11:08:39.560554+0000","last_active":"2023-11-28T11:08:39.564343+0000","last_peered":"2023-11-28T11:08:39.564343+0000","last_clean":"2023-11-28T11:08:39.564343+0000","last_became_active":"2023-11-28T11:08:39.560032+0000","last_became_peered":"2023-11-28T11:08:39.560032+0000","last_unstale":"2023-11-28T11:08:39.564343+0000","last_undegraded":"2023-11-28T11:08:39.564343+0000","last_fullsized":"2023-11-28T11:08:39.564343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551526+0000","last_change":"2023-11-28T11:08:35.146846+0000","last_active":"2023-11-28T11:08:39.551526+0000","last_peered":"2023-11-28T11:08:39.551526+0000","last_clean":"2023-11-28T11:08:39.551526+0000","last_became_active":"2023-11-28T11:08:35.146565+0000","last_became_peered":"2023-11-28T11:08:35.146565+0000","last_unstale":"2023-11-28T11:08:39.551526+0000","last_undegraded":"2023-11-28T11:08:39.551526+0000","last_fullsized":"2023-11-28T11:08:39.551526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545959+0000","last_change":"2023-11-28T11:08:35.537655+0000","last_active":"2023-11-28T11:08:39.545959+0000","last_peered":"2023-11-28T11:08:39.545959+0000","last_clean":"2023-11-28T11:08:39.545959+0000","last_became_active":"2023-11-28T11:08:35.537401+0000","last_became_peered":"2023-11-28T11:08:35.537401+0000","last_unstale":"2023-11-28T11:08:39.545959+0000","last_undegraded":"2023-11-28T11:08:39.545959+0000","last_fullsized":"2023-11-28T11:08:39.545959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.559490+0000","last_change":"2023-11-28T11:08:39.558756+0000","last_active":"2023-11-28T11:08:39.559490+0000","last_peered":"2023-11-28T11:08:39.559490+0000","last_clean":"2023-11-28T11:08:39.559490+0000","last_became_active":"2023-11-28T11:08:39.558382+0000","last_became_peered":"2023-11-28T11:08:39.558382+0000","last_unstale":"2023-11-28T11:08:39.559490+0000","last_undegraded":"2023-11-28T11:08:39.559490+0000","last_fullsized":"2023-11-28T11:08:39.559490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:08.685769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549379+0000","last_change":"2023-11-28T11:08:35.143155+0000","last_active":"2023-11-28T11:08:39.549379+0000","last_peered":"2023-11-28T11:08:39.549379+0000","last_clean":"2023-11-28T11:08:39.549379+0000","last_became_active":"2023-11-28T11:08:35.142856+0000","last_became_peered":"2023-11-28T11:08:35.142856+0000","last_unstale":"2023-11-28T11:08:39.549379+0000","last_undegraded":"2023-11-28T11:08:39.549379+0000","last_fullsized":"2023-11-28T11:08:39.549379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546595+0000","last_change":"2023-11-28T11:08:35.550010+0000","last_active":"2023-11-28T11:08:39.546595+0000","last_peered":"2023-11-28T11:08:39.546595+0000","last_clean":"2023-11-28T11:08:39.546595+0000","last_became_active":"2023-11-28T11:08:35.548764+0000","last_became_peered":"2023-11-28T11:08:35.548764+0000","last_unstale":"2023-11-28T11:08:39.546595+0000","last_undegraded":"2023-11-28T11:08:39.546595+0000","last_fullsized":"2023-11-28T11:08:39.546595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568175+0000","last_change":"2023-11-28T11:08:39.562607+0000","last_active":"2023-11-28T11:08:39.568175+0000","last_peered":"2023-11-28T11:08:39.568175+0000","last_clean":"2023-11-28T11:08:39.568175+0000","last_became_active":"2023-11-28T11:08:39.562144+0000","last_became_peered":"2023-11-28T11:08:39.562144+0000","last_unstale":"2023-11-28T11:08:39.568175+0000","last_undegraded":"2023-11-28T11:08:39.568175+0000","last_fullsized":"2023-11-28T11:08:39.568175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:43:13.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549488+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.549488+0000","last_peered":"2023-11-28T11:08:39.549488+0000","last_clean":"2023-11-28T11:08:39.549488+0000","last_became_active":"2023-11-28T11:08:35.146468+0000","last_became_peered":"2023-11-28T11:08:35.146468+0000","last_unstale":"2023-11-28T11:08:39.549488+0000","last_undegraded":"2023-11-28T11:08:39.549488+0000","last_fullsized":"2023-11-28T11:08:39.549488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:42.348852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552146+0000","last_change":"2023-11-28T11:08:35.545831+0000","last_active":"2023-11-28T11:08:39.552146+0000","last_peered":"2023-11-28T11:08:39.552146+0000","last_clean":"2023-11-28T11:08:39.552146+0000","last_became_active":"2023-11-28T11:08:35.544509+0000","last_became_peered":"2023-11-28T11:08:35.544509+0000","last_unstale":"2023-11-28T11:08:39.552146+0000","last_undegraded":"2023-11-28T11:08:39.552146+0000","last_fullsized":"2023-11-28T11:08:39.552146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564184+0000","last_change":"2023-11-28T11:08:39.560957+0000","last_active":"2023-11-28T11:08:39.564184+0000","last_peered":"2023-11-28T11:08:39.564184+0000","last_clean":"2023-11-28T11:08:39.564184+0000","last_became_active":"2023-11-28T11:08:39.560011+0000","last_became_peered":"2023-11-28T11:08:39.560011+0000","last_unstale":"2023-11-28T11:08:39.564184+0000","last_undegraded":"2023-11-28T11:08:39.564184+0000","last_fullsized":"2023-11-28T11:08:39.564184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551361+0000","last_change":"2023-11-28T11:08:35.137486+0000","last_active":"2023-11-28T11:08:39.551361+0000","last_peered":"2023-11-28T11:08:39.551361+0000","last_clean":"2023-11-28T11:08:39.551361+0000","last_became_active":"2023-11-28T11:08:35.137192+0000","last_became_peered":"2023-11-28T11:08:35.137192+0000","last_unstale":"2023-11-28T11:08:39.551361+0000","last_undegraded":"2023-11-28T11:08:39.551361+0000","last_fullsized":"2023-11-28T11:08:39.551361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:09.971307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545264+0000","last_change":"2023-11-28T11:08:35.543758+0000","last_active":"2023-11-28T11:08:39.545264+0000","last_peered":"2023-11-28T11:08:39.545264+0000","last_clean":"2023-11-28T11:08:39.545264+0000","last_became_active":"2023-11-28T11:08:35.543535+0000","last_became_peered":"2023-11-28T11:08:35.543535+0000","last_unstale":"2023-11-28T11:08:39.545264+0000","last_undegraded":"2023-11-28T11:08:39.545264+0000","last_fullsized":"2023-11-28T11:08:39.545264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570639+0000","last_change":"2023-11-28T11:08:39.567742+0000","last_active":"2023-11-28T11:08:39.570639+0000","last_peered":"2023-11-28T11:08:39.570639+0000","last_clean":"2023-11-28T11:08:39.570639+0000","last_became_active":"2023-11-28T11:08:39.567313+0000","last_became_peered":"2023-11-28T11:08:39.567313+0000","last_unstale":"2023-11-28T11:08:39.570639+0000","last_undegraded":"2023-11-28T11:08:39.570639+0000","last_fullsized":"2023-11-28T11:08:39.570639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552255+0000","last_change":"2023-11-28T11:08:35.142255+0000","last_active":"2023-11-28T11:08:39.552255+0000","last_peered":"2023-11-28T11:08:39.552255+0000","last_clean":"2023-11-28T11:08:39.552255+0000","last_became_active":"2023-11-28T11:08:35.140243+0000","last_became_peered":"2023-11-28T11:08:35.140243+0000","last_unstale":"2023-11-28T11:08:39.552255+0000","last_undegraded":"2023-11-28T11:08:39.552255+0000","last_fullsized":"2023-11-28T11:08:39.552255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:43.134431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546061+0000","last_change":"2023-11-28T11:08:35.540595+0000","last_active":"2023-11-28T11:08:39.546061+0000","last_peered":"2023-11-28T11:08:39.546061+0000","last_clean":"2023-11-28T11:08:39.546061+0000","last_became_active":"2023-11-28T11:08:35.540303+0000","last_became_peered":"2023-11-28T11:08:35.540303+0000","last_unstale":"2023-11-28T11:08:39.546061+0000","last_undegraded":"2023-11-28T11:08:39.546061+0000","last_fullsized":"2023-11-28T11:08:39.546061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562667+0000","last_change":"2023-11-28T11:08:39.560601+0000","last_active":"2023-11-28T11:08:39.562667+0000","last_peered":"2023-11-28T11:08:39.562667+0000","last_clean":"2023-11-28T11:08:39.562667+0000","last_became_active":"2023-11-28T11:08:39.560147+0000","last_became_peered":"2023-11-28T11:08:39.560147+0000","last_unstale":"2023-11-28T11:08:39.562667+0000","last_undegraded":"2023-11-28T11:08:39.562667+0000","last_fullsized":"2023-11-28T11:08:39.562667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:39.607355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552185+0000","last_change":"2023-11-28T11:08:35.142804+0000","last_active":"2023-11-28T11:08:39.552185+0000","last_peered":"2023-11-28T11:08:39.552185+0000","last_clean":"2023-11-28T11:08:39.552185+0000","last_became_active":"2023-11-28T11:08:35.142407+0000","last_became_peered":"2023-11-28T11:08:35.142407+0000","last_unstale":"2023-11-28T11:08:39.552185+0000","last_undegraded":"2023-11-28T11:08:39.552185+0000","last_fullsized":"2023-11-28T11:08:39.552185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:12.791046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560104+0000","last_change":"2023-11-28T11:08:35.543448+0000","last_active":"2023-11-28T11:08:39.560104+0000","last_peered":"2023-11-28T11:08:39.560104+0000","last_clean":"2023-11-28T11:08:39.560104+0000","last_became_active":"2023-11-28T11:08:35.541094+0000","last_became_peered":"2023-11-28T11:08:35.541094+0000","last_unstale":"2023-11-28T11:08:39.560104+0000","last_undegraded":"2023-11-28T11:08:39.560104+0000","last_fullsized":"2023-11-28T11:08:39.560104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571651+0000","last_change":"2023-11-28T11:08:39.562935+0000","last_active":"2023-11-28T11:08:39.571651+0000","last_peered":"2023-11-28T11:08:39.571651+0000","last_clean":"2023-11-28T11:08:39.571651+0000","last_became_active":"2023-11-28T11:08:39.562529+0000","last_became_peered":"2023-11-28T11:08:39.562529+0000","last_unstale":"2023-11-28T11:08:39.571651+0000","last_undegraded":"2023-11-28T11:08:39.571651+0000","last_fullsized":"2023-11-28T11:08:39.571651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:31.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552244+0000","last_change":"2023-11-28T11:08:35.143035+0000","last_active":"2023-11-28T11:08:39.552244+0000","last_peered":"2023-11-28T11:08:39.552244+0000","last_clean":"2023-11-28T11:08:39.552244+0000","last_became_active":"2023-11-28T11:08:35.142697+0000","last_became_peered":"2023-11-28T11:08:35.142697+0000","last_unstale":"2023-11-28T11:08:39.552244+0000","last_undegraded":"2023-11-28T11:08:39.552244+0000","last_fullsized":"2023-11-28T11:08:39.552244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:24.895027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552639+0000","last_change":"2023-11-28T11:08:35.535083+0000","last_active":"2023-11-28T11:08:39.552639+0000","last_peered":"2023-11-28T11:08:39.552639+0000","last_clean":"2023-11-28T11:08:39.552639+0000","last_became_active":"2023-11-28T11:08:35.534800+0000","last_became_peered":"2023-11-28T11:08:35.534800+0000","last_unstale":"2023-11-28T11:08:39.552639+0000","last_undegraded":"2023-11-28T11:08:39.552639+0000","last_fullsized":"2023-11-28T11:08:39.552639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571669+0000","last_change":"2023-11-28T11:08:39.565208+0000","last_active":"2023-11-28T11:08:39.571669+0000","last_peered":"2023-11-28T11:08:39.571669+0000","last_clean":"2023-11-28T11:08:39.571669+0000","last_became_active":"2023-11-28T11:08:39.564299+0000","last_became_peered":"2023-11-28T11:08:39.564299+0000","last_unstale":"2023-11-28T11:08:39.571669+0000","last_undegraded":"2023-11-28T11:08:39.571669+0000","last_fullsized":"2023-11-28T11:08:39.571669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:48.486752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546343+0000","last_change":"2023-11-28T11:08:35.141725+0000","last_active":"2023-11-28T11:08:39.546343+0000","last_peered":"2023-11-28T11:08:39.546343+0000","last_clean":"2023-11-28T11:08:39.546343+0000","last_became_active":"2023-11-28T11:08:35.141445+0000","last_became_peered":"2023-11-28T11:08:35.141445+0000","last_unstale":"2023-11-28T11:08:39.546343+0000","last_undegraded":"2023-11-28T11:08:39.546343+0000","last_fullsized":"2023-11-28T11:08:39.546343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556057+0000","last_change":"2023-11-28T11:08:35.548819+0000","last_active":"2023-11-28T11:08:39.556057+0000","last_peered":"2023-11-28T11:08:39.556057+0000","last_clean":"2023-11-28T11:08:39.556057+0000","last_became_active":"2023-11-28T11:08:35.548403+0000","last_became_peered":"2023-11-28T11:08:35.548403+0000","last_unstale":"2023-11-28T11:08:39.556057+0000","last_undegraded":"2023-11-28T11:08:39.556057+0000","last_fullsized":"2023-11-28T11:08:39.556057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571605+0000","last_change":"2023-11-28T11:08:39.562941+0000","last_active":"2023-11-28T11:08:39.571605+0000","last_peered":"2023-11-28T11:08:39.571605+0000","last_clean":"2023-11-28T11:08:39.571605+0000","last_became_active":"2023-11-28T11:08:39.562555+0000","last_became_peered":"2023-11-28T11:08:39.562555+0000","last_unstale":"2023-11-28T11:08:39.571605+0000","last_undegraded":"2023-11-28T11:08:39.571605+0000","last_fullsized":"2023-11-28T11:08:39.571605+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544942+0000","last_change":"2023-11-28T11:08:35.142283+0000","last_active":"2023-11-28T11:08:39.544942+0000","last_peered":"2023-11-28T11:08:39.544942+0000","last_clean":"2023-11-28T11:08:39.544942+0000","last_became_active":"2023-11-28T11:08:35.141976+0000","last_became_peered":"2023-11-28T11:08:35.141976+0000","last_unstale":"2023-11-28T11:08:39.544942+0000","last_undegraded":"2023-11-28T11:08:39.544942+0000","last_fullsized":"2023-11-28T11:08:39.544942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556613+0000","last_change":"2023-11-28T11:08:35.540882+0000","last_active":"2023-11-28T11:08:39.556613+0000","last_peered":"2023-11-28T11:08:39.556613+0000","last_clean":"2023-11-28T11:08:39.556613+0000","last_became_active":"2023-11-28T11:08:35.540566+0000","last_became_peered":"2023-11-28T11:08:35.540566+0000","last_unstale":"2023-11-28T11:08:39.556613+0000","last_undegraded":"2023-11-28T11:08:39.556613+0000","last_fullsized":"2023-11-28T11:08:39.556613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570782+0000","last_change":"2023-11-28T11:08:39.566349+0000","last_active":"2023-11-28T11:08:39.570782+0000","last_peered":"2023-11-28T11:08:39.570782+0000","last_clean":"2023-11-28T11:08:39.570782+0000","last_became_active":"2023-11-28T11:08:39.565923+0000","last_became_peered":"2023-11-28T11:08:39.565923+0000","last_unstale":"2023-11-28T11:08:39.570782+0000","last_undegraded":"2023-11-28T11:08:39.570782+0000","last_fullsized":"2023-11-28T11:08:39.570782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551243+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.551243+0000","last_peered":"2023-11-28T11:08:39.551243+0000","last_clean":"2023-11-28T11:08:39.551243+0000","last_became_active":"2023-11-28T11:08:35.146466+0000","last_became_peered":"2023-11-28T11:08:35.146466+0000","last_unstale":"2023-11-28T11:08:39.551243+0000","last_undegraded":"2023-11-28T11:08:39.551243+0000","last_fullsized":"2023-11-28T11:08:39.551243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556761+0000","last_change":"2023-11-28T11:08:35.540325+0000","last_active":"2023-11-28T11:08:39.556761+0000","last_peered":"2023-11-28T11:08:39.556761+0000","last_clean":"2023-11-28T11:08:39.556761+0000","last_became_active":"2023-11-28T11:08:35.540045+0000","last_became_peered":"2023-11-28T11:08:35.540045+0000","last_unstale":"2023-11-28T11:08:39.556761+0000","last_undegraded":"2023-11-28T11:08:39.556761+0000","last_fullsized":"2023-11-28T11:08:39.556761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566338+0000","last_change":"2023-11-28T11:08:39.562874+0000","last_active":"2023-11-28T11:08:39.566338+0000","last_peered":"2023-11-28T11:08:39.566338+0000","last_clean":"2023-11-28T11:08:39.566338+0000","last_became_active":"2023-11-28T11:08:39.562484+0000","last_became_peered":"2023-11-28T11:08:39.562484+0000","last_unstale":"2023-11-28T11:08:39.566338+0000","last_undegraded":"2023-11-28T11:08:39.566338+0000","last_fullsized":"2023-11-28T11:08:39.566338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552660+0000","last_change":"2023-11-28T11:08:35.139637+0000","last_active":"2023-11-28T11:08:39.552660+0000","last_peered":"2023-11-28T11:08:39.552660+0000","last_clean":"2023-11-28T11:08:39.552660+0000","last_became_active":"2023-11-28T11:08:35.138268+0000","last_became_peered":"2023-11-28T11:08:35.138268+0000","last_unstale":"2023-11-28T11:08:39.552660+0000","last_undegraded":"2023-11-28T11:08:39.552660+0000","last_fullsized":"2023-11-28T11:08:39.552660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556539+0000","last_change":"2023-11-28T11:08:35.543314+0000","last_active":"2023-11-28T11:08:39.556539+0000","last_peered":"2023-11-28T11:08:39.556539+0000","last_clean":"2023-11-28T11:08:39.556539+0000","last_became_active":"2023-11-28T11:08:35.539838+0000","last_became_peered":"2023-11-28T11:08:35.539838+0000","last_unstale":"2023-11-28T11:08:39.556539+0000","last_undegraded":"2023-11-28T11:08:39.556539+0000","last_fullsized":"2023-11-28T11:08:39.556539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:53.884226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568846+0000","last_change":"2023-11-28T11:08:39.565918+0000","last_active":"2023-11-28T11:08:39.568846+0000","last_peered":"2023-11-28T11:08:39.568846+0000","last_clean":"2023-11-28T11:08:39.568846+0000","last_became_active":"2023-11-28T11:08:39.562911+0000","last_became_peered":"2023-11-28T11:08:39.562911+0000","last_unstale":"2023-11-28T11:08:39.568846+0000","last_undegraded":"2023-11-28T11:08:39.568846+0000","last_fullsized":"2023-11-28T11:08:39.568846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:11:09.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546326+0000","last_change":"2023-11-28T11:08:35.143397+0000","last_active":"2023-11-28T11:08:39.546326+0000","last_peered":"2023-11-28T11:08:39.546326+0000","last_clean":"2023-11-28T11:08:39.546326+0000","last_became_active":"2023-11-28T11:08:35.142466+0000","last_became_peered":"2023-11-28T11:08:35.142466+0000","last_unstale":"2023-11-28T11:08:39.546326+0000","last_undegraded":"2023-11-28T11:08:39.546326+0000","last_fullsized":"2023-11-28T11:08:39.546326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556168+0000","last_change":"2023-11-28T11:08:35.534626+0000","last_active":"2023-11-28T11:08:39.556168+0000","last_peered":"2023-11-28T11:08:39.556168+0000","last_clean":"2023-11-28T11:08:39.556168+0000","last_became_active":"2023-11-28T11:08:35.534379+0000","last_became_peered":"2023-11-28T11:08:35.534379+0000","last_unstale":"2023-11-28T11:08:39.556168+0000","last_undegraded":"2023-11-28T11:08:39.556168+0000","last_fullsized":"2023-11-28T11:08:39.556168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573390+0000","last_change":"2023-11-28T11:08:39.569661+0000","last_active":"2023-11-28T11:08:39.573390+0000","last_peered":"2023-11-28T11:08:39.573390+0000","last_clean":"2023-11-28T11:08:39.573390+0000","last_became_active":"2023-11-28T11:08:39.569175+0000","last_became_peered":"2023-11-28T11:08:39.569175+0000","last_unstale":"2023-11-28T11:08:39.573390+0000","last_undegraded":"2023-11-28T11:08:39.573390+0000","last_fullsized":"2023-11-28T11:08:39.573390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:59.446120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544982+0000","last_change":"2023-11-28T11:08:35.143399+0000","last_active":"2023-11-28T11:08:39.544982+0000","last_peered":"2023-11-28T11:08:39.544982+0000","last_clean":"2023-11-28T11:08:39.544982+0000","last_became_active":"2023-11-28T11:08:35.143102+0000","last_became_peered":"2023-11-28T11:08:35.143102+0000","last_unstale":"2023-11-28T11:08:39.544982+0000","last_undegraded":"2023-11-28T11:08:39.544982+0000","last_fullsized":"2023-11-28T11:08:39.544982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556842+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:08:39.556842+0000","last_peered":"2023-11-28T11:08:39.556842+0000","last_clean":"2023-11-28T11:08:39.556842+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:08:39.556842+0000","last_undegraded":"2023-11-28T11:08:39.556842+0000","last_fullsized":"2023-11-28T11:08:39.556842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545915+0000","last_change":"2023-11-28T11:08:35.541344+0000","last_active":"2023-11-28T11:08:39.545915+0000","last_peered":"2023-11-28T11:08:39.545915+0000","last_clean":"2023-11-28T11:08:39.545915+0000","last_became_active":"2023-11-28T11:08:35.541086+0000","last_became_peered":"2023-11-28T11:08:35.541086+0000","last_unstale":"2023-11-28T11:08:39.545915+0000","last_undegraded":"2023-11-28T11:08:39.545915+0000","last_fullsized":"2023-11-28T11:08:39.545915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.572142+0000","last_change":"2023-11-28T11:08:39.567333+0000","last_active":"2023-11-28T11:08:39.572142+0000","last_peered":"2023-11-28T11:08:39.572142+0000","last_clean":"2023-11-28T11:08:39.572142+0000","last_became_active":"2023-11-28T11:08:39.566952+0000","last_became_peered":"2023-11-28T11:08:39.566952+0000","last_unstale":"2023-11-28T11:08:39.572142+0000","last_undegraded":"2023-11-28T11:08:39.572142+0000","last_fullsized":"2023-11-28T11:08:39.572142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545875+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:08:39.545875+0000","last_peered":"2023-11-28T11:08:39.545875+0000","last_clean":"2023-11-28T11:08:39.545875+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:08:39.545875+0000","last_undegraded":"2023-11-28T11:08:39.545875+0000","last_fullsized":"2023-11-28T11:08:39.545875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:40.959549+0000","last_change":"2023-11-28T11:08:35.138175+0000","last_active":"2023-11-28T11:08:40.959549+0000","last_peered":"2023-11-28T11:08:40.959549+0000","last_clean":"2023-11-28T11:08:40.959549+0000","last_became_active":"2023-11-28T11:08:35.137577+0000","last_became_peered":"2023-11-28T11:08:35.137577+0000","last_unstale":"2023-11-28T11:08:40.959549+0000","last_undegraded":"2023-11-28T11:08:40.959549+0000","last_fullsized":"2023-11-28T11:08:40.959549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560017+0000","last_change":"2023-11-28T11:08:35.540521+0000","last_active":"2023-11-28T11:08:39.560017+0000","last_peered":"2023-11-28T11:08:39.560017+0000","last_clean":"2023-11-28T11:08:39.560017+0000","last_became_active":"2023-11-28T11:08:35.538206+0000","last_became_peered":"2023-11-28T11:08:35.538206+0000","last_unstale":"2023-11-28T11:08:39.560017+0000","last_undegraded":"2023-11-28T11:08:39.560017+0000","last_fullsized":"2023-11-28T11:08:39.560017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568578+0000","last_change":"2023-11-28T11:08:39.565755+0000","last_active":"2023-11-28T11:08:39.568578+0000","last_peered":"2023-11-28T11:08:39.568578+0000","last_clean":"2023-11-28T11:08:39.568578+0000","last_became_active":"2023-11-28T11:08:39.565093+0000","last_became_peered":"2023-11-28T11:08:39.565093+0000","last_unstale":"2023-11-28T11:08:39.568578+0000","last_undegraded":"2023-11-28T11:08:39.568578+0000","last_fullsized":"2023-11-28T11:08:39.568578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555643+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:08:39.555643+0000","last_peered":"2023-11-28T11:08:39.555643+0000","last_clean":"2023-11-28T11:08:39.555643+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:08:39.555643+0000","last_undegraded":"2023-11-28T11:08:39.555643+0000","last_fullsized":"2023-11-28T11:08:39.555643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550860+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:08:39.550860+0000","last_peered":"2023-11-28T11:08:39.550860+0000","last_clean":"2023-11-28T11:08:39.550860+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:08:39.550860+0000","last_undegraded":"2023-11-28T11:08:39.550860+0000","last_fullsized":"2023-11-28T11:08:39.550860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549175+0000","last_change":"2023-11-28T11:08:35.146578+0000","last_active":"2023-11-28T11:08:39.549175+0000","last_peered":"2023-11-28T11:08:39.549175+0000","last_clean":"2023-11-28T11:08:39.549175+0000","last_became_active":"2023-11-28T11:08:35.146300+0000","last_became_peered":"2023-11-28T11:08:35.146300+0000","last_unstale":"2023-11-28T11:08:39.549175+0000","last_undegraded":"2023-11-28T11:08:39.549175+0000","last_fullsized":"2023-11-28T11:08:39.549175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552070+0000","last_change":"2023-11-28T11:08:35.545787+0000","last_active":"2023-11-28T11:08:39.552070+0000","last_peered":"2023-11-28T11:08:39.552070+0000","last_clean":"2023-11-28T11:08:39.552070+0000","last_became_active":"2023-11-28T11:08:35.544359+0000","last_became_peered":"2023-11-28T11:08:35.544359+0000","last_unstale":"2023-11-28T11:08:39.552070+0000","last_undegraded":"2023-11-28T11:08:39.552070+0000","last_fullsized":"2023-11-28T11:08:39.552070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.561183+0000","last_change":"2023-11-28T11:08:39.558516+0000","last_active":"2023-11-28T11:08:39.561183+0000","last_peered":"2023-11-28T11:08:39.561183+0000","last_clean":"2023-11-28T11:08:39.561183+0000","last_became_active":"2023-11-28T11:08:39.558117+0000","last_became_peered":"2023-11-28T11:08:39.558117+0000","last_unstale":"2023-11-28T11:08:39.561183+0000","last_undegraded":"2023-11-28T11:08:39.561183+0000","last_fullsized":"2023-11-28T11:08:39.561183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551550+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:08:39.551550+0000","last_peered":"2023-11-28T11:08:39.551550+0000","last_clean":"2023-11-28T11:08:39.551550+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:08:39.551550+0000","last_undegraded":"2023-11-28T11:08:39.551550+0000","last_fullsized":"2023-11-28T11:08:39.551550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.554459+0000","last_change":"2023-11-28T11:08:35.134990+0000","last_active":"2023-11-28T11:08:39.554459+0000","last_peered":"2023-11-28T11:08:39.554459+0000","last_clean":"2023-11-28T11:08:39.554459+0000","last_became_active":"2023-11-28T11:08:35.134728+0000","last_became_peered":"2023-11-28T11:08:35.134728+0000","last_unstale":"2023-11-28T11:08:39.554459+0000","last_undegraded":"2023-11-28T11:08:39.554459+0000","last_fullsized":"2023-11-28T11:08:39.554459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556117+0000","last_change":"2023-11-28T11:08:35.540730+0000","last_active":"2023-11-28T11:08:39.556117+0000","last_peered":"2023-11-28T11:08:39.556117+0000","last_clean":"2023-11-28T11:08:39.556117+0000","last_became_active":"2023-11-28T11:08:35.540312+0000","last_became_peered":"2023-11-28T11:08:35.540312+0000","last_unstale":"2023-11-28T11:08:39.556117+0000","last_undegraded":"2023-11-28T11:08:39.556117+0000","last_fullsized":"2023-11-28T11:08:39.556117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568532+0000","last_change":"2023-11-28T11:08:39.561818+0000","last_active":"2023-11-28T11:08:39.568532+0000","last_peered":"2023-11-28T11:08:39.568532+0000","last_clean":"2023-11-28T11:08:39.568532+0000","last_became_active":"2023-11-28T11:08:39.561292+0000","last_became_peered":"2023-11-28T11:08:39.561292+0000","last_unstale":"2023-11-28T11:08:39.568532+0000","last_undegraded":"2023-11-28T11:08:39.568532+0000","last_fullsized":"2023-11-28T11:08:39.568532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550685+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:08:39.550685+0000","last_peered":"2023-11-28T11:08:39.550685+0000","last_clean":"2023-11-28T11:08:39.550685+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:08:39.550685+0000","last_undegraded":"2023-11-28T11:08:39.550685+0000","last_fullsized":"2023-11-28T11:08:39.550685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546311+0000","last_change":"2023-11-28T11:08:35.140783+0000","last_active":"2023-11-28T11:08:39.546311+0000","last_peered":"2023-11-28T11:08:39.546311+0000","last_clean":"2023-11-28T11:08:39.546311+0000","last_became_active":"2023-11-28T11:08:35.140512+0000","last_became_peered":"2023-11-28T11:08:35.140512+0000","last_unstale":"2023-11-28T11:08:39.546311+0000","last_undegraded":"2023-11-28T11:08:39.546311+0000","last_fullsized":"2023-11-28T11:08:39.546311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551352+0000","last_change":"2023-11-28T11:08:35.547824+0000","last_active":"2023-11-28T11:08:39.551352+0000","last_peered":"2023-11-28T11:08:39.551352+0000","last_clean":"2023-11-28T11:08:39.551352+0000","last_became_active":"2023-11-28T11:08:35.547517+0000","last_became_peered":"2023-11-28T11:08:35.547517+0000","last_unstale":"2023-11-28T11:08:39.551352+0000","last_undegraded":"2023-11-28T11:08:39.551352+0000","last_fullsized":"2023-11-28T11:08:39.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:55.715375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570503+0000","last_change":"2023-11-28T11:08:39.564906+0000","last_active":"2023-11-28T11:08:39.570503+0000","last_peered":"2023-11-28T11:08:39.570503+0000","last_clean":"2023-11-28T11:08:39.570503+0000","last_became_active":"2023-11-28T11:08:39.564258+0000","last_became_peered":"2023-11-28T11:08:39.564258+0000","last_unstale":"2023-11-28T11:08:39.570503+0000","last_undegraded":"2023-11-28T11:08:39.570503+0000","last_fullsized":"2023-11-28T11:08:39.570503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546086+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:08:39.546086+0000","last_peered":"2023-11-28T11:08:39.546086+0000","last_clean":"2023-11-28T11:08:39.546086+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:08:39.546086+0000","last_undegraded":"2023-11-28T11:08:39.546086+0000","last_fullsized":"2023-11-28T11:08:39.546086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556375+0000","last_change":"2023-11-28T11:08:35.144005+0000","last_active":"2023-11-28T11:08:39.556375+0000","last_peered":"2023-11-28T11:08:39.556375+0000","last_clean":"2023-11-28T11:08:39.556375+0000","last_became_active":"2023-11-28T11:08:35.141190+0000","last_became_peered":"2023-11-28T11:08:35.141190+0000","last_unstale":"2023-11-28T11:08:39.556375+0000","last_undegraded":"2023-11-28T11:08:39.556375+0000","last_fullsized":"2023-11-28T11:08:39.556375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555324+0000","last_change":"2023-11-28T11:08:35.539604+0000","last_active":"2023-11-28T11:08:39.555324+0000","last_peered":"2023-11-28T11:08:39.555324+0000","last_clean":"2023-11-28T11:08:39.555324+0000","last_became_active":"2023-11-28T11:08:35.539319+0000","last_became_peered":"2023-11-28T11:08:35.539319+0000","last_unstale":"2023-11-28T11:08:39.555324+0000","last_undegraded":"2023-11-28T11:08:39.555324+0000","last_fullsized":"2023-11-28T11:08:39.555324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562105+0000","last_change":"2023-11-28T11:08:39.559157+0000","last_active":"2023-11-28T11:08:39.562105+0000","last_peered":"2023-11-28T11:08:39.562105+0000","last_clean":"2023-11-28T11:08:39.562105+0000","last_became_active":"2023-11-28T11:08:39.558790+0000","last_became_peered":"2023-11-28T11:08:39.558790+0000","last_unstale":"2023-11-28T11:08:39.562105+0000","last_undegraded":"2023-11-28T11:08:39.562105+0000","last_fullsized":"2023-11-28T11:08:39.562105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:18.469803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550458+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:08:39.550458+0000","last_peered":"2023-11-28T11:08:39.550458+0000","last_clean":"2023-11-28T11:08:39.550458+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:08:39.550458+0000","last_undegraded":"2023-11-28T11:08:39.550458+0000","last_fullsized":"2023-11-28T11:08:39.550458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555493+0000","last_change":"2023-11-28T11:08:35.136293+0000","last_active":"2023-11-28T11:08:39.555493+0000","last_peered":"2023-11-28T11:08:39.555493+0000","last_clean":"2023-11-28T11:08:39.555493+0000","last_became_active":"2023-11-28T11:08:35.136011+0000","last_became_peered":"2023-11-28T11:08:35.136011+0000","last_unstale":"2023-11-28T11:08:39.555493+0000","last_undegraded":"2023-11-28T11:08:39.555493+0000","last_fullsized":"2023-11-28T11:08:39.555493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556312+0000","last_change":"2023-11-28T11:08:35.548876+0000","last_active":"2023-11-28T11:08:39.556312+0000","last_peered":"2023-11-28T11:08:39.556312+0000","last_clean":"2023-11-28T11:08:39.556312+0000","last_became_active":"2023-11-28T11:08:35.548606+0000","last_became_peered":"2023-11-28T11:08:35.548606+0000","last_unstale":"2023-11-28T11:08:39.556312+0000","last_undegraded":"2023-11-28T11:08:39.556312+0000","last_fullsized":"2023-11-28T11:08:39.556312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570495+0000","last_change":"2023-11-28T11:08:39.564356+0000","last_active":"2023-11-28T11:08:39.570495+0000","last_peered":"2023-11-28T11:08:39.570495+0000","last_clean":"2023-11-28T11:08:39.570495+0000","last_became_active":"2023-11-28T11:08:39.563998+0000","last_became_peered":"2023-11-28T11:08:39.563998+0000","last_unstale":"2023-11-28T11:08:39.570495+0000","last_undegraded":"2023-11-28T11:08:39.570495+0000","last_fullsized":"2023-11-28T11:08:39.570495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553351+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:08:39.553351+0000","last_peered":"2023-11-28T11:08:39.553351+0000","last_clean":"2023-11-28T11:08:39.553351+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:08:39.553351+0000","last_undegraded":"2023-11-28T11:08:39.553351+0000","last_fullsized":"2023-11-28T11:08:39.553351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546501+0000","last_change":"2023-11-28T11:08:35.141912+0000","last_active":"2023-11-28T11:08:39.546501+0000","last_peered":"2023-11-28T11:08:39.546501+0000","last_clean":"2023-11-28T11:08:39.546501+0000","last_became_active":"2023-11-28T11:08:35.141586+0000","last_became_peered":"2023-11-28T11:08:35.141586+0000","last_unstale":"2023-11-28T11:08:39.546501+0000","last_undegraded":"2023-11-28T11:08:39.546501+0000","last_fullsized":"2023-11-28T11:08:39.546501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552908+0000","last_change":"2023-11-28T11:08:35.542267+0000","last_active":"2023-11-28T11:08:39.552908+0000","last_peered":"2023-11-28T11:08:39.552908+0000","last_clean":"2023-11-28T11:08:39.552908+0000","last_became_active":"2023-11-28T11:08:35.541323+0000","last_became_peered":"2023-11-28T11:08:35.541323+0000","last_unstale":"2023-11-28T11:08:39.552908+0000","last_undegraded":"2023-11-28T11:08:39.552908+0000","last_fullsized":"2023-11-28T11:08:39.552908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570775+0000","last_change":"2023-11-28T11:08:39.566246+0000","last_active":"2023-11-28T11:08:39.570775+0000","last_peered":"2023-11-28T11:08:39.570775+0000","last_clean":"2023-11-28T11:08:39.570775+0000","last_became_active":"2023-11-28T11:08:39.565895+0000","last_became_peered":"2023-11-28T11:08:39.565895+0000","last_unstale":"2023-11-28T11:08:39.570775+0000","last_undegraded":"2023-11-28T11:08:39.570775+0000","last_fullsized":"2023-11-28T11:08:39.570775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556229+0000","last_change":"2023-11-28T11:08:35.146576+0000","last_active":"2023-11-28T11:08:39.556229+0000","last_peered":"2023-11-28T11:08:39.556229+0000","last_clean":"2023-11-28T11:08:39.556229+0000","last_became_active":"2023-11-28T11:08:35.146283+0000","last_became_peered":"2023-11-28T11:08:35.146283+0000","last_unstale":"2023-11-28T11:08:39.556229+0000","last_undegraded":"2023-11-28T11:08:39.556229+0000","last_fullsized":"2023-11-28T11:08:39.556229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542850+0000","last_change":"2023-11-28T11:08:35.536348+0000","last_active":"2023-11-28T11:08:39.542850+0000","last_peered":"2023-11-28T11:08:39.542850+0000","last_clean":"2023-11-28T11:08:39.542850+0000","last_became_active":"2023-11-28T11:08:35.536095+0000","last_became_peered":"2023-11-28T11:08:35.536095+0000","last_unstale":"2023-11-28T11:08:39.542850+0000","last_undegraded":"2023-11-28T11:08:39.542850+0000","last_fullsized":"2023-11-28T11:08:39.542850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.991268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570654+0000","last_change":"2023-11-28T11:08:39.567740+0000","last_active":"2023-11-28T11:08:39.570654+0000","last_peered":"2023-11-28T11:08:39.570654+0000","last_clean":"2023-11-28T11:08:39.570654+0000","last_became_active":"2023-11-28T11:08:39.567279+0000","last_became_peered":"2023-11-28T11:08:39.567279+0000","last_unstale":"2023-11-28T11:08:39.570654+0000","last_undegraded":"2023-11-28T11:08:39.570654+0000","last_fullsized":"2023-11-28T11:08:39.570654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546145+0000","last_change":"2023-11-28T11:08:35.141111+0000","last_active":"2023-11-28T11:08:39.546145+0000","last_peered":"2023-11-28T11:08:39.546145+0000","last_clean":"2023-11-28T11:08:39.546145+0000","last_became_active":"2023-11-28T11:08:35.140804+0000","last_became_peered":"2023-11-28T11:08:35.140804+0000","last_unstale":"2023-11-28T11:08:39.546145+0000","last_undegraded":"2023-11-28T11:08:39.546145+0000","last_fullsized":"2023-11-28T11:08:39.546145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556281+0000","last_change":"2023-11-28T11:08:35.547330+0000","last_active":"2023-11-28T11:08:39.556281+0000","last_peered":"2023-11-28T11:08:39.556281+0000","last_clean":"2023-11-28T11:08:39.556281+0000","last_became_active":"2023-11-28T11:08:35.546970+0000","last_became_peered":"2023-11-28T11:08:35.546970+0000","last_unstale":"2023-11-28T11:08:39.556281+0000","last_undegraded":"2023-11-28T11:08:39.556281+0000","last_fullsized":"2023-11-28T11:08:39.556281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:28.259978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568054+0000","last_change":"2023-11-28T11:08:39.565657+0000","last_active":"2023-11-28T11:08:39.568054+0000","last_peered":"2023-11-28T11:08:39.568054+0000","last_clean":"2023-11-28T11:08:39.568054+0000","last_became_active":"2023-11-28T11:08:39.565004+0000","last_became_peered":"2023-11-28T11:08:39.565004+0000","last_unstale":"2023-11-28T11:08:39.568054+0000","last_undegraded":"2023-11-28T11:08:39.568054+0000","last_fullsized":"2023-11-28T11:08:39.568054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546414+0000","last_change":"2023-11-28T11:08:35.146886+0000","last_active":"2023-11-28T11:08:39.546414+0000","last_peered":"2023-11-28T11:08:39.546414+0000","last_clean":"2023-11-28T11:08:39.546414+0000","last_became_active":"2023-11-28T11:08:35.146600+0000","last_became_peered":"2023-11-28T11:08:35.146600+0000","last_unstale":"2023-11-28T11:08:39.546414+0000","last_undegraded":"2023-11-28T11:08:39.546414+0000","last_fullsized":"2023-11-28T11:08:39.546414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556350+0000","last_change":"2023-11-28T11:08:35.538328+0000","last_active":"2023-11-28T11:08:39.556350+0000","last_peered":"2023-11-28T11:08:39.556350+0000","last_clean":"2023-11-28T11:08:39.556350+0000","last_became_active":"2023-11-28T11:08:35.537863+0000","last_became_peered":"2023-11-28T11:08:35.537863+0000","last_unstale":"2023-11-28T11:08:39.556350+0000","last_undegraded":"2023-11-28T11:08:39.556350+0000","last_fullsized":"2023-11-28T11:08:39.556350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566546+0000","last_change":"2023-11-28T11:08:39.562424+0000","last_active":"2023-11-28T11:08:39.566546+0000","last_peered":"2023-11-28T11:08:39.566546+0000","last_clean":"2023-11-28T11:08:39.566546+0000","last_became_active":"2023-11-28T11:08:39.562065+0000","last_became_peered":"2023-11-28T11:08:39.562065+0000","last_unstale":"2023-11-28T11:08:39.566546+0000","last_undegraded":"2023-11-28T11:08:39.566546+0000","last_fullsized":"2023-11-28T11:08:39.566546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:39.095254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544783+0000","last_change":"2023-11-28T11:08:35.138855+0000","last_active":"2023-11-28T11:08:39.544783+0000","last_peered":"2023-11-28T11:08:39.544783+0000","last_clean":"2023-11-28T11:08:39.544783+0000","last_became_active":"2023-11-28T11:08:35.138526+0000","last_became_peered":"2023-11-28T11:08:35.138526+0000","last_unstale":"2023-11-28T11:08:39.544783+0000","last_undegraded":"2023-11-28T11:08:39.544783+0000","last_fullsized":"2023-11-28T11:08:39.544783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544513+0000","last_change":"2023-11-28T11:08:35.539629+0000","last_active":"2023-11-28T11:08:39.544513+0000","last_peered":"2023-11-28T11:08:39.544513+0000","last_clean":"2023-11-28T11:08:39.544513+0000","last_became_active":"2023-11-28T11:08:35.534961+0000","last_became_peered":"2023-11-28T11:08:35.534961+0000","last_unstale":"2023-11-28T11:08:39.544513+0000","last_undegraded":"2023-11-28T11:08:39.544513+0000","last_fullsized":"2023-11-28T11:08:39.544513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571346+0000","last_change":"2023-11-28T11:08:39.570212+0000","last_active":"2023-11-28T11:08:39.571346+0000","last_peered":"2023-11-28T11:08:39.571346+0000","last_clean":"2023-11-28T11:08:39.571346+0000","last_became_active":"2023-11-28T11:08:39.569900+0000","last_became_peered":"2023-11-28T11:08:39.569900+0000","last_unstale":"2023-11-28T11:08:39.571346+0000","last_undegraded":"2023-11-28T11:08:39.571346+0000","last_fullsized":"2023-11-28T11:08:39.571346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552849+0000","last_change":"2023-11-28T11:08:35.139415+0000","last_active":"2023-11-28T11:08:39.552849+0000","last_peered":"2023-11-28T11:08:39.552849+0000","last_clean":"2023-11-28T11:08:39.552849+0000","last_became_active":"2023-11-28T11:08:35.139152+0000","last_became_peered":"2023-11-28T11:08:35.139152+0000","last_unstale":"2023-11-28T11:08:39.552849+0000","last_undegraded":"2023-11-28T11:08:39.552849+0000","last_fullsized":"2023-11-28T11:08:39.552849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552466+0000","last_change":"2023-11-28T11:08:35.544319+0000","last_active":"2023-11-28T11:08:39.552466+0000","last_peered":"2023-11-28T11:08:39.552466+0000","last_clean":"2023-11-28T11:08:39.552466+0000","last_became_active":"2023-11-28T11:08:35.544032+0000","last_became_peered":"2023-11-28T11:08:35.544032+0000","last_unstale":"2023-11-28T11:08:39.552466+0000","last_undegraded":"2023-11-28T11:08:39.552466+0000","last_fullsized":"2023-11-28T11:08:39.552466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:36.189248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551454+0000","last_change":"2023-11-28T11:08:35.534208+0000","last_active":"2023-11-28T11:08:39.551454+0000","last_peered":"2023-11-28T11:08:39.551454+0000","last_clean":"2023-11-28T11:08:39.551454+0000","last_became_active":"2023-11-28T11:08:35.533929+0000","last_became_peered":"2023-11-28T11:08:35.533929+0000","last_unstale":"2023-11-28T11:08:39.551454+0000","last_undegraded":"2023-11-28T11:08:39.551454+0000","last_fullsized":"2023-11-28T11:08:39.551454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:44.139630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544658+0000","last_change":"2023-11-28T11:08:35.540664+0000","last_active":"2023-11-28T11:08:39.544658+0000","last_peered":"2023-11-28T11:08:39.544658+0000","last_clean":"2023-11-28T11:08:39.544658+0000","last_became_active":"2023-11-28T11:08:35.540243+0000","last_became_peered":"2023-11-28T11:08:35.540243+0000","last_unstale":"2023-11-28T11:08:39.544658+0000","last_undegraded":"2023-11-28T11:08:39.544658+0000","last_fullsized":"2023-11-28T11:08:39.544658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556410+0000","last_change":"2023-11-28T11:08:35.544372+0000","last_active":"2023-11-28T11:08:39.556410+0000","last_peered":"2023-11-28T11:08:39.556410+0000","last_clean":"2023-11-28T11:08:39.556410+0000","last_became_active":"2023-11-28T11:08:35.543867+0000","last_became_peered":"2023-11-28T11:08:35.543867+0000","last_unstale":"2023-11-28T11:08:39.556410+0000","last_undegraded":"2023-11-28T11:08:39.556410+0000","last_fullsized":"2023-11-28T11:08:39.556410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:30.327766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544832+0000","last_change":"2023-11-28T11:08:35.543075+0000","last_active":"2023-11-28T11:08:39.544832+0000","last_peered":"2023-11-28T11:08:39.544832+0000","last_clean":"2023-11-28T11:08:39.544832+0000","last_became_active":"2023-11-28T11:08:35.542645+0000","last_became_peered":"2023-11-28T11:08:35.542645+0000","last_unstale":"2023-11-28T11:08:39.544832+0000","last_undegraded":"2023-11-28T11:08:39.544832+0000","last_fullsized":"2023-11-28T11:08:39.544832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555411+0000","last_change":"2023-11-28T11:08:35.546225+0000","last_active":"2023-11-28T11:08:39.555411+0000","last_peered":"2023-11-28T11:08:39.555411+0000","last_clean":"2023-11-28T11:08:39.555411+0000","last_became_active":"2023-11-28T11:08:35.545970+0000","last_became_peered":"2023-11-28T11:08:35.545970+0000","last_unstale":"2023-11-28T11:08:39.555411+0000","last_undegraded":"2023-11-28T11:08:39.555411+0000","last_fullsized":"2023-11-28T11:08:39.555411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4387,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9248,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362592,"statfs":{"total":96636764160,"available":96627294208,"internally_reserved":0,"allocated":876544,"data_stored":698919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8060,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363780,"statfs":{"total":96636764160,"available":96628510720,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8128,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363712,"statfs":{"total":96636764160,"available":96628441088,"internally_reserved":0,"allocated":253952,"data_stored":103935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8064,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363776,"statfs":{"total":96636764160,"available":96628506624,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8132,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363708,"statfs":{"total":96636764160,"available":96628436992,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8212,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363628,"statfs":{"total":96636764160,"available":96628355072,"internally_reserved":0,"allocated":274432,"data_stored":108391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":105366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:53.311 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:08:53.312 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T11:08:53.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T11:08:53.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T11:08:53.313 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T11:08:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 Processor -- start 2023-11-28T11:08:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 -- start start 2023-11-28T11:08:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff63415cd10 0x7ff6340ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6340b1fb0 0x7ff6340cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6340cbfb0 con 0x7ff63415d0e0 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63406cee0 con 0x7ff6340b1fb0 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.465+0000 7ff63c405700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340cb710 con 0x7ff63415cd10 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6340b1fb0 0x7ff6340cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6340b1fb0 0x7ff6340cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48708/0 (socket says 172.21.15.83:48708) 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 -- 172.21.15.83:0/3915144986 learned_addr learned my addr 172.21.15.83:0/3915144986 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a1a1700 1 --2- 172.21.15.83:0/3915144986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff63415cd10 0x7ff6340ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 -- 172.21.15.83:0/3915144986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff63415cd10 msgr2=0x7ff6340ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 --2- 172.21.15.83:0/3915144986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff63415cd10 0x7ff6340ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 -- 172.21.15.83:0/3915144986 >> v1:172.21.15.90:6789/0 conn(0x7ff63415d0e0 legacy=0x7ff6340b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 -- 172.21.15.83:0/3915144986 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340cb4e0 con 0x7ff6340b1fb0 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63a9a2700 1 --2- 172.21.15.83:0/3915144986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6340b1fb0 0x7ff6340cafa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009e50 tx=0x7ff62c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3653be808eec473 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63919f700 1 -- 172.21.15.83:0/3915144986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00e070 con 0x7ff6340b1fb0 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63919f700 1 -- 172.21.15.83:0/3915144986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c00b9c0 con 0x7ff6340b1fb0 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.466+0000 7ff63919f700 1 -- 172.21.15.83:0/3915144986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00bd70 con 0x7ff6340b1fb0 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 -- 172.21.15.83:0/3915144986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6340b1fb0 msgr2=0x7ff6340cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 --2- 172.21.15.83:0/3915144986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6340b1fb0 0x7ff6340cafa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009e50 tx=0x7ff62c00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 -- 172.21.15.83:0/3915144986 shutdown_connections 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 --2- 172.21.15.83:0/3915144986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6340b1fb0 0x7ff6340cafa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 --2- 172.21.15.83:0/3915144986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff63415cd10 0x7ff6340ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 -- 172.21.15.83:0/3915144986 >> 172.21.15.83:0/3915144986 conn(0x7ff63406e9a0 msgr2=0x7ff6340cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 -- 172.21.15.83:0/3915144986 shutdown_connections 2023-11-28T11:08:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.467+0000 7ff63c405700 1 -- 172.21.15.83:0/3915144986 wait complete. 2023-11-28T11:08:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.468+0000 7ff63c405700 1 Processor -- start 2023-11-28T11:08:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.468+0000 7ff63c405700 1 -- start start 2023-11-28T11:08:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff6340b1fb0 0x7ff63414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff63415cd10 0x7ff6341f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 0x7ff63414a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340ad890 con 0x7ff63415d0e0 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340ad280 con 0x7ff63415cd10 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341f24b0 con 0x7ff6340b1fb0 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 0x7ff63414a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 0x7ff63414a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48714/0 (socket says 172.21.15.83:48714) 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 -- 172.21.15.83:0/687158022 learned_addr learned my addr 172.21.15.83:0/687158022 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff6399a0700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff63415cd10 0x7ff6341f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a1a1700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff6340b1fb0 0x7ff63414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 -- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff6340b1fb0 msgr2=0x7ff63414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff6340b1fb0 0x7ff63414b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 -- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff63415cd10 msgr2=0x7ff6341f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff63415cd10 0x7ff6341f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 -- 172.21.15.83:0/687158022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c009b30 con 0x7ff63415d0e0 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63a9a2700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 0x7ff63414a980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff63406dc40 tx=0x7ff62c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004800 con 0x7ff63415d0e0 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c004960 con 0x7ff63415d0e0 2023-11-28T11:08:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c01c680 con 0x7ff63415d0e0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6340cbcd0 con 0x7ff63415d0e0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.469+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6340cb9f0 con 0x7ff63415d0e0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.470+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63406b9b0 con 0x7ff63415d0e0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.471+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff62c02c070 con 0x7ff63415d0e0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.471+0000 7ff626ffd700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff6180682f0 0x7ff61806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.471+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff62c087930 con 0x7ff63415d0e0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.471+0000 7ff63a1a1700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff6180682f0 0x7ff61806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.472+0000 7ff63a1a1700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff6180682f0 0x7ff61806a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff628006b20 tx=0x7ff628008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.475+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff62c053ea0 con 0x7ff63415d0e0 2023-11-28T11:08:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.709+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff634074320 con 0x7ff63415d0e0 2023-11-28T11:08:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.712+0000 7ff626ffd700 1 -- 172.21.15.83:0/687158022 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166379 (secure 0 0 0) 0x7ff62c0574c0 con 0x7ff63415d0e0 2023-11-28T11:08:53.716 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff6180682f0 msgr2=0x7ff61806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff6180682f0 0x7ff61806a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff628006b20 tx=0x7ff628008040 comp rx=0 tx=0).stop 2023-11-28T11:08:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 msgr2=0x7ff63414a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 0x7ff63414a980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff63406dc40 tx=0x7ff62c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 shutdown_connections 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff6180682f0 0x7ff61806a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63415d0e0 0x7ff63414a980 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff6340b1fb0 0x7ff63414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 --2- 172.21.15.83:0/687158022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff63415cd10 0x7ff6341f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 >> 172.21.15.83:0/687158022 conn(0x7ff63406e9a0 msgr2=0x7ff6340af880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 shutdown_connections 2023-11-28T11:08:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.715+0000 7ff63c405700 1 -- 172.21.15.83:0/687158022 wait complete. 2023-11-28T11:08:53.728 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":103263},{"type":"v1","addr":"172.21.15.83:6833","nonce":103263}]},"active_addr":"172.21.15.83:6833/103263","active_change":"2023-11-28T11:08:23.560068+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":4280971076}]},{"addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":4060038285}]},{"addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2906722979}]},{"addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2049669641}]}]} 2023-11-28T11:08:53.732 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T11:08:53.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T11:08:53.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.852+0000 7f2393be3700 1 Processor -- start 2023-11-28T11:08:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.852+0000 7f2393be3700 1 -- start start 2023-11-28T11:08:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f2393be3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f238c0cf0d0 0x7f238c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f2393be3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0b18a0 0x7f238c0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f2393be3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c0cf540 con 0x7f238c159830 2023-11-28T11:08:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f2393be3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238c15d480 con 0x7f238c0cf0d0 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f2393be3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0b2260 con 0x7f238c0b18a0 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239117e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f238c0cf0d0 0x7f238c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239197f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f238c159830 0x7f238c15de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46578/0 (socket says 172.21.15.83:46578) 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239197f700 1 -- 172.21.15.83:0/1178809384 learned_addr learned my addr 172.21.15.83:0/1178809384 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f2392180700 1 --2- 172.21.15.83:0/1178809384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0b18a0 0x7f238c0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239117e700 1 -- 172.21.15.83:0/1178809384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0b18a0 msgr2=0x7f238c0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239117e700 1 --2- 172.21.15.83:0/1178809384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0b18a0 0x7f238c0b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239117e700 1 -- 172.21.15.83:0/1178809384 >> v1:172.21.15.83:6789/0 conn(0x7f238c159830 legacy=0x7f238c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.853+0000 7f239097d700 1 -- 172.21.15.83:0/1178809384 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425608223 0 0) 0x7f238c0cf540 con 0x7f238c159830 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f239117e700 1 -- 172.21.15.83:0/1178809384 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c0cf790 con 0x7f238c0cf0d0 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f239117e700 1 --2- 172.21.15.83:0/1178809384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0cf0d0 0x7f238c06c5f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f237c009840 tx=0x7f237c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583818313499b3e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f239097d700 1 -- 172.21.15.83:0/1178809384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c011830 con 0x7f238c0cf0d0 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f239097d700 1 -- 172.21.15.83:0/1178809384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c011990 con 0x7f238c0cf0d0 2023-11-28T11:08:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f239097d700 1 -- 172.21.15.83:0/1178809384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c011d20 con 0x7f238c0cf0d0 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f2393be3700 1 -- 172.21.15.83:0/1178809384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0cf0d0 msgr2=0x7f238c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f2393be3700 1 --2- 172.21.15.83:0/1178809384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0cf0d0 0x7f238c06c5f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f237c009840 tx=0x7f237c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f2393be3700 1 -- 172.21.15.83:0/1178809384 shutdown_connections 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f2393be3700 1 --2- 172.21.15.83:0/1178809384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0b18a0 0x7f238c0b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f2393be3700 1 --2- 172.21.15.83:0/1178809384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0cf0d0 0x7f238c06c5f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.854+0000 7f2393be3700 1 -- 172.21.15.83:0/1178809384 >> 172.21.15.83:0/1178809384 conn(0x7f238c06e9a0 msgr2=0x7f238c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.855+0000 7f2393be3700 1 -- 172.21.15.83:0/1178809384 shutdown_connections 2023-11-28T11:08:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.855+0000 7f2393be3700 1 -- 172.21.15.83:0/1178809384 wait complete. 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 Processor -- start 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 -- start start 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 0x7f238c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0cf0d0 0x7f238c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f238c159830 0x7f238c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0c98e0 con 0x7f238c0b18a0 2023-11-28T11:08:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0ca240 con 0x7f238c159830 2023-11-28T11:08:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.856+0000 7f2393be3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0c8c60 con 0x7f238c0cf0d0 2023-11-28T11:08:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 0x7f238c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 0x7f238c0c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48724/0 (socket says 172.21.15.83:48724) 2023-11-28T11:08:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 -- 172.21.15.83:0/246126102 learned_addr learned my addr 172.21.15.83:0/246126102 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f2392180700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f238c159830 0x7f238c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239117e700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0cf0d0 0x7f238c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 -- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0cf0d0 msgr2=0x7f238c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0cf0d0 0x7f238c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 -- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f238c159830 msgr2=0x7f238c1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f238c159830 0x7f238c1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 -- 172.21.15.83:0/246126102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c0094a0 con 0x7f238c0b18a0 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f239197f700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 0x7f238c0c8660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f23880011e0 tx=0x7f238800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388004c80 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388004e00 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c15d690 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388007310 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.857+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c0cfa70 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.858+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238c15d970 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.858+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f238801a020 con 0x7f238c0b18a0 2023-11-28T11:08:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.859+0000 7f23827fc700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f23700682f0 0x7f237006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.859+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2388085a70 con 0x7f238c0b18a0 2023-11-28T11:08:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.859+0000 7f239117e700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f23700682f0 0x7f237006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.859+0000 7f239117e700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f23700682f0 0x7f237006a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f237c0090b0 tx=0x7f237c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:53.863+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2388051fe0 con 0x7f238c0b18a0 2023-11-28T11:08:54.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.046+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f238c074320 con 0x7f238c0b18a0 2023-11-28T11:08:54.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.047+0000 7f23827fc700 1 -- 172.21.15.83:0/246126102 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f2388055600 con 0x7f238c0b18a0 2023-11-28T11:08:54.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:54.051 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:54.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f23700682f0 msgr2=0x7f237006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f23700682f0 0x7f237006a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f237c0090b0 tx=0x7f237c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:54.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 msgr2=0x7f238c0c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 0x7f238c0c8660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f23880011e0 tx=0x7f238800ca30 comp rx=0 tx=0).stop 2023-11-28T11:08:54.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 shutdown_connections 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f238c159830 0x7f238c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f23700682f0 0x7f237006a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f238c0b18a0 0x7f238c0c8660 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 --2- 172.21.15.83:0/246126102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f238c0cf0d0 0x7f238c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 >> 172.21.15.83:0/246126102 conn(0x7f238c06e9a0 msgr2=0x7f238c156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 shutdown_connections 2023-11-28T11:08:54.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.049+0000 7f2393be3700 1 -- 172.21.15.83:0/246126102 wait complete. 2023-11-28T11:08:54.063 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T11:08:54.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.180+0000 7f5baf270700 1 Processor -- start 2023-11-28T11:08:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.181+0000 7f5baf270700 1 -- start start 2023-11-28T11:08:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5baf270700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5ba815cd10 0x7f5ba80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5baf270700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5baf270700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80cbfb0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5baf270700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba806cee0 con 0x7f5ba815cd10 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5baf270700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80cb710 con 0x7f5ba815d0e0 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad00c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5ba815cd10 0x7f5ba80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bac80b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad80d700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f5ba80b18a0 0x7f5ba81609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44942/0 (socket says 172.21.15.83:44942) 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad80d700 1 -- 172.21.15.83:0/1234213592 learned_addr learned my addr 172.21.15.83:0/1234213592 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad00c700 1 -- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 msgr2=0x7f5ba80cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad00c700 1 --2- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad00c700 1 -- 172.21.15.83:0/1234213592 >> v1:172.21.15.90:6789/0 conn(0x7f5ba80b18a0 legacy=0x7f5ba81609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad00c700 1 -- 172.21.15.83:0/1234213592 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80cb4e0 con 0x7f5ba815cd10 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5bad00c700 1 --2- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba80ca970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc06e9a1e3a7d49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.182+0000 7f5b9ffff700 1 -- 172.21.15.83:0/1234213592 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000e070 con 0x7f5ba815cd10 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5b9ffff700 1 -- 172.21.15.83:0/1234213592 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba000bb20 con 0x7f5ba815cd10 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5b9ffff700 1 -- 172.21.15.83:0/1234213592 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0012430 con 0x7f5ba815cd10 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 -- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 msgr2=0x7f5ba80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 --2- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba80ca970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 -- 172.21.15.83:0/1234213592 shutdown_connections 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 --2- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba80ca970 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 --2- 172.21.15.83:0/1234213592 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 -- 172.21.15.83:0/1234213592 >> 172.21.15.83:0/1234213592 conn(0x7f5ba806e9a0 msgr2=0x7f5ba80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 -- 172.21.15.83:0/1234213592 shutdown_connections 2023-11-28T11:08:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.183+0000 7f5baf270700 1 -- 172.21.15.83:0/1234213592 wait complete. 2023-11-28T11:08:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.184+0000 7f5baf270700 1 Processor -- start 2023-11-28T11:08:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.184+0000 7f5baf270700 1 -- start start 2023-11-28T11:08:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5baf270700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 0x7f5ba81f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5baf270700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba81efc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5baf270700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5baf270700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81f2750 con 0x7f5ba80b18a0 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5baf270700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81f6c90 con 0x7f5ba815cd10 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5baf270700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81f0160 con 0x7f5ba815d0e0 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5bad00c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 0x7f5ba81f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5bad00c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 0x7f5ba81f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48740/0 (socket says 172.21.15.83:48740) 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5bad00c700 1 -- 172.21.15.83:0/3132572595 learned_addr learned my addr 172.21.15.83:0/3132572595 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5bac80b700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba81efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.185+0000 7f5bad80d700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5bad00c700 1 -- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 msgr2=0x7f5ba80b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5bad00c700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5bad00c700 1 -- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 msgr2=0x7f5ba81efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5bad00c700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba81efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5bad00c700 1 -- 172.21.15.83:0/3132572595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba0009a70 con 0x7f5ba80b18a0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5bad00c700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 0x7f5ba81f6750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0014040 tx=0x7f5ba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0012430 con 0x7f5ba80b18a0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0020070 con 0x7f5ba80b18a0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba80cbcd0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0004280 con 0x7f5ba80b18a0 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.186+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba80cb9f0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.187+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba806b9b0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.187+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ba00043e0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.188+0000 7f5b9dffb700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b8c0682f0 0x7f5b8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.188+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ba0022070 con 0x7f5ba80b18a0 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.188+0000 7f5bac80b700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b8c0682f0 0x7f5b8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.188+0000 7f5bac80b700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b8c0682f0 0x7f5b8c06a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80c3350 tx=0x7f5b98009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.192+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ba00547e0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.375+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ba80c6ed0 con 0x7f5ba80b18a0 2023-11-28T11:08:54.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.377+0000 7f5b9dffb700 1 -- 172.21.15.83:0/3132572595 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f5ba0057e00 con 0x7f5ba80b18a0 2023-11-28T11:08:54.378 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:54.379 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:54.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b8c0682f0 msgr2=0x7f5b8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b8c0682f0 0x7f5b8c06a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80c3350 tx=0x7f5b98009210 comp rx=0 tx=0).stop 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 msgr2=0x7f5ba81f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 0x7f5ba81f6750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0014040 tx=0x7f5ba0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 shutdown_connections 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ba815d0e0 0x7f5ba80b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b8c0682f0 0x7f5b8c06a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80b18a0 0x7f5ba81f6750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 --2- 172.21.15.83:0/3132572595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ba815cd10 0x7f5ba81efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 >> 172.21.15.83:0/3132572595 conn(0x7f5ba806e9a0 msgr2=0x7f5ba80cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 shutdown_connections 2023-11-28T11:08:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.379+0000 7f5baf270700 1 -- 172.21.15.83:0/3132572595 wait complete. 2023-11-28T11:08:54.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T11:08:54.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T11:08:54.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T11:08:54.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T11:08:54.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T11:08:54.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T11:08:54.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T11:08:54.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T11:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.596+0000 7f6bb088c700 1 Processor -- start 2023-11-28T11:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.596+0000 7f6bb088c700 1 -- start start 2023-11-28T11:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.597+0000 7f6bb088c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6bac0cc0f0 0x7f6bac0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.597+0000 7f6bb088c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.597+0000 7f6bb088c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0cca00 con 0x7f6bac15d3b0 2023-11-28T11:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.597+0000 7f6bb088c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac160930 con 0x7f6bac0cc0f0 2023-11-28T11:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.597+0000 7f6bb088c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac160c10 con 0x7f6bac160150 2023-11-28T11:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:32948/0 (socket says 172.21.15.83:32948) 2023-11-28T11:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/2362478025 learned_addr learned my addr 172.21.15.83:0/2362478025 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6baa59c700 1 --2- 172.21.15.83:0/2362478025 >> v2:172.21.15.90:3300/0 conn(0x7f6bac0cc0f0 0x7f6bac0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/2362478025 >> v2:172.21.15.90:3300/0 conn(0x7f6bac0cc0f0 msgr2=0x7f6bac0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 --2- 172.21.15.83:0/2362478025 >> v2:172.21.15.90:3300/0 conn(0x7f6bac0cc0f0 0x7f6bac0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/2362478025 >> v1:172.21.15.83:6789/0 conn(0x7f6bac15d3b0 legacy=0x7f6bac0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/2362478025 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac160ef0 con 0x7f6bac160150 2023-11-28T11:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.598+0000 7f6ba9d9b700 1 --2- 172.21.15.83:0/2362478025 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0cc4c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0098f0 tx=0x7f6b9c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e0f2002edefd66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.599+0000 7f6ba959a700 1 -- 172.21.15.83:0/2362478025 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c004230 con 0x7f6bac160150 2023-11-28T11:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.599+0000 7f6ba959a700 1 -- 172.21.15.83:0/2362478025 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c004390 con 0x7f6bac160150 2023-11-28T11:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.599+0000 7f6ba959a700 1 -- 172.21.15.83:0/2362478025 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0036a0 con 0x7f6bac160150 2023-11-28T11:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.599+0000 7f6bb088c700 1 -- 172.21.15.83:0/2362478025 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 msgr2=0x7f6bac0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.599+0000 7f6bb088c700 1 --2- 172.21.15.83:0/2362478025 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0cc4c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0098f0 tx=0x7f6b9c009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.600+0000 7f6bb088c700 1 -- 172.21.15.83:0/2362478025 shutdown_connections 2023-11-28T11:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.600+0000 7f6bb088c700 1 --2- 172.21.15.83:0/2362478025 >> v2:172.21.15.90:3300/0 conn(0x7f6bac0cc0f0 0x7f6bac0cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.600+0000 7f6bb088c700 1 --2- 172.21.15.83:0/2362478025 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0cc4c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.600+0000 7f6bb088c700 1 -- 172.21.15.83:0/2362478025 >> 172.21.15.83:0/2362478025 conn(0x7f6bac0b1090 msgr2=0x7f6bac0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.600+0000 7f6bb088c700 1 -- 172.21.15.83:0/2362478025 shutdown_connections 2023-11-28T11:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.600+0000 7f6bb088c700 1 -- 172.21.15.83:0/2362478025 wait complete. 2023-11-28T11:08:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.601+0000 7f6bb088c700 1 Processor -- start 2023-11-28T11:08:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.601+0000 7f6bb088c700 1 -- start start 2023-11-28T11:08:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7f6bb088c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bac0cc0f0 0x7f6bac1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7f6bb088c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bac15d3b0 0x7f6bac158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7f6bb088c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7f6bb088c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac15ebe0 con 0x7f6bac0cc0f0 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7f6bb088c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b3a00 con 0x7f6bac15d3b0 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7f6bb088c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b3ce0 con 0x7f6bac160150 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6baa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bac0cc0f0 0x7f6bac1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6baa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bac0cc0f0 0x7f6bac1528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48752/0 (socket says 172.21.15.83:48752) 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6baa59c700 1 -- 172.21.15.83:0/1461010988 learned_addr learned my addr 172.21.15.83:0/1461010988 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6ba9d9b700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bac15d3b0 0x7f6bac158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6baad9d700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7fb22cc73700 1 Processor -- start 2023-11-28T11:08:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.602+0000 7fb22cc73700 1 -- start start 2023-11-28T11:08:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 msgr2=0x7f6bac0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6ba9d9b700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bac160150 0x7f6bac0b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bac0cc0f0 msgr2=0x7f6bac1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6ba9d9b700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bac0cc0f0 0x7f6bac1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7f6ba9d9b700 1 -- 172.21.15.83:0/1461010988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c009550 con 0x7f6bac15d3b0 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22cc73700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb22816cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22cc73700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb2280712a0 0x7fb2280c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22cc73700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb2280cc750 0x7fb2280ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22cc73700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22816de80 con 0x7fb2280cc750 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22cc73700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb228089db0 con 0x7fb2280712a0 2023-11-28T11:08:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22cc73700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280cd0e0 con 0x7fb2280899e0 2023-11-28T11:08:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.603+0000 7fb22659c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb22816cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f6ba9d9b700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bac15d3b0 0x7f6bac158d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c015000 tx=0x7f6b9c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb22816cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:32960/0 (socket says 172.21.15.83:32960) 2023-11-28T11:08:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 -- 172.21.15.83:0/2257243746 learned_addr learned my addr 172.21.15.83:0/2257243746 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb226d9d700 1 --2- 172.21.15.83:0/2257243746 >> v2:172.21.15.83:3300/0 conn(0x7fb2280cc750 0x7fb2280ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 -- 172.21.15.83:0/2257243746 >> v2:172.21.15.90:3300/0 conn(0x7fb2280712a0 msgr2=0x7fb2280c91d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 --2- 172.21.15.83:0/2257243746 >> v2:172.21.15.90:3300/0 conn(0x7fb2280712a0 0x7fb2280c91d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 -- 172.21.15.83:0/2257243746 >> v2:172.21.15.83:3300/0 conn(0x7fb2280cc750 msgr2=0x7fb2280ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 --2- 172.21.15.83:0/2257243746 >> v2:172.21.15.83:3300/0 conn(0x7fb2280cc750 0x7fb2280ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7fb22659c700 1 -- 172.21.15.83:0/2257243746 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228169860 con 0x7fb2280899e0 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0045d0 con 0x7f6bac15d3b0 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c00b660 con 0x7f6bac15d3b0 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00b9d0 con 0x7f6bac15d3b0 2023-11-28T11:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac15cb60 con 0x7f6bac15d3b0 2023-11-28T11:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac15ce90 con 0x7f6bac15d3b0 2023-11-28T11:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.604+0000 7f0ec853a700 1 Processor -- start 2023-11-28T11:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.605+0000 7fb22659c700 1 --2- 172.21.15.83:0/2257243746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb22816cf50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2280cf290 tx=0x7fb218009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a28d9bad76dfbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.605+0000 7f0ec853a700 1 -- start start 2023-11-28T11:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.605+0000 7ff8a11f0700 1 Processor -- start 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.605+0000 7fb22559a700 1 -- 172.21.15.83:0/2257243746 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218011830 con 0x7fb2280899e0 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.605+0000 7fb22559a700 1 -- 172.21.15.83:0/2257243746 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218011990 con 0x7fb2280899e0 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.605+0000 7fb22559a700 1 -- 172.21.15.83:0/2257243746 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218011d20 con 0x7fb2280899e0 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7fb22cc73700 1 -- 172.21.15.83:0/2257243746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 msgr2=0x7fb22816cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7fb22cc73700 1 --2- 172.21.15.83:0/2257243746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb22816cf50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2280cf290 tx=0x7fb218009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7fb22cc73700 1 -- 172.21.15.83:0/2257243746 shutdown_connections 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7fb22cc73700 1 --2- 172.21.15.83:0/2257243746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb22816cf50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7fb22cc73700 1 --2- 172.21.15.83:0/2257243746 >> v2:172.21.15.83:3300/0 conn(0x7fb2280cc750 0x7fb2280ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7fb22cc73700 1 --2- 172.21.15.83:0/2257243746 >> v2:172.21.15.90:3300/0 conn(0x7fb2280712a0 0x7fb2280c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7fb22cc73700 1 -- 172.21.15.83:0/2257243746 >> 172.21.15.83:0/2257243746 conn(0x7fb2280b1380 msgr2=0x7fb2280ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b9c017020 con 0x7f6bac15d3b0 2023-11-28T11:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f6b9affd700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6b8c068340 0x7f6b8c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6b9c0877e0 con 0x7f6bac15d3b0 2023-11-28T11:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f6baa59c700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6b8c068340 0x7f6b8c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7ff8a11f0700 1 -- start start 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7f0ec853a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ec00cf290 0x7f0ec006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7f0ec853a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0ec0158520 0x7f0ec0070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7f0ec853a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec006d7d0 con 0x7f0ec015cd90 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7f0ec853a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec006d180 con 0x7f0ec00cf290 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.606+0000 7f0ec853a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec006d460 con 0x7f0ec0158520 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec5ad5700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f0ec015cd90 0x7f0ec006dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:44972/0 (socket says 172.21.15.83:44972) 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec5ad5700 1 -- 172.21.15.83:0/4213284129 learned_addr learned my addr 172.21.15.83:0/4213284129 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec62d6700 1 --2- 172.21.15.83:0/4213284129 >> v2:172.21.15.83:3300/0 conn(0x7f0ec00cf290 0x7f0ec006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec62d6700 1 -- 172.21.15.83:0/4213284129 >> v2:172.21.15.90:3300/0 conn(0x7f0ec0158520 msgr2=0x7f0ec0070ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec62d6700 1 --2- 172.21.15.83:0/4213284129 >> v2:172.21.15.90:3300/0 conn(0x7f0ec0158520 0x7f0ec0070ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec62d6700 1 -- 172.21.15.83:0/4213284129 >> v1:172.21.15.90:6789/0 conn(0x7f0ec015cd90 legacy=0x7f0ec006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec62d6700 1 -- 172.21.15.83:0/4213284129 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec015d1e0 con 0x7f0ec00cf290 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.607+0000 7f0ec62d6700 1 --2- 172.21.15.83:0/4213284129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec006e440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00ae5a0 tx=0x7f0ebc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba01c319ab5b7f81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7f0ec52d4700 1 -- 172.21.15.83:0/4213284129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc011830 con 0x7f0ec00cf290 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7f0ec52d4700 1 -- 172.21.15.83:0/4213284129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc011990 con 0x7f0ec00cf290 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7f0ec52d4700 1 -- 172.21.15.83:0/4213284129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc011d00 con 0x7f0ec00cf290 2023-11-28T11:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7f6baa59c700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6b8c068340 0x7f6b8c06a7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0007af0 tx=0x7f6ba0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7ff8a11f0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff89404fb90 0x7ff894054470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7ff8a11f0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89404f830 con 0x7ff8940531e0 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7ff8a11f0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89404f2b0 con 0x7ff894052700 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7ff8a11f0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940df0c0 con 0x7ff89404fb90 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7f0ec853a700 1 -- 172.21.15.83:0/4213284129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 msgr2=0x7f0ec006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.608+0000 7f0ec853a700 1 --2- 172.21.15.83:0/4213284129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec006e440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00ae5a0 tx=0x7f0ebc009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff8a09ef700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff8940531e0 0x7ff8940e8730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46636/0 (socket says 172.21.15.83:46636) 2023-11-28T11:08:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff8a09ef700 1 -- 172.21.15.83:0/1414703268 learned_addr learned my addr 172.21.15.83:0/1414703268 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f0ec853a700 1 -- 172.21.15.83:0/4213284129 shutdown_connections 2023-11-28T11:08:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f0ec853a700 1 --2- 172.21.15.83:0/4213284129 >> v2:172.21.15.90:3300/0 conn(0x7f0ec0158520 0x7f0ec0070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f0ec853a700 1 --2- 172.21.15.83:0/4213284129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec006e440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f0ec853a700 1 -- 172.21.15.83:0/4213284129 >> 172.21.15.83:0/4213284129 conn(0x7f0ec00b1380 msgr2=0x7f0ec00ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f6bb088c700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f6bac0b2a70 0x7f6bac06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 --> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bac0b3e20 con 0x7f6bac0b2a70 2023-11-28T11:08:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89b7fe700 1 --2- 172.21.15.83:0/1414703268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff89404fb90 0x7ff894054470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f6baad9d700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f6bac0b2a70 0x7f6bac06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89affd700 1 -- 172.21.15.83:0/1414703268 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657903746 0 0) 0x7ff89404f830 con 0x7ff8940531e0 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89affd700 1 -- 172.21.15.83:0/1414703268 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff88c0035b0 con 0x7ff8940531e0 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89b7fe700 1 -- 172.21.15.83:0/1414703268 >> v1:172.21.15.90:6789/0 conn(0x7ff894052700 legacy=0x7ff894052ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89b7fe700 1 -- 172.21.15.83:0/1414703268 >> v1:172.21.15.83:6789/0 conn(0x7ff8940531e0 legacy=0x7ff8940e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89b7fe700 1 -- 172.21.15.83:0/1414703268 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89403a450 con 0x7ff89404fb90 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89b7fe700 1 --2- 172.21.15.83:0/1414703268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff89404fb90 0x7ff894054470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff890009870 tx=0x7ff890009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce819f7a0616eb94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7effa6c6c700 1 Processor -- start 2023-11-28T11:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7effa6c6c700 1 -- start start 2023-11-28T11:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7f6baad9d700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f6bac0b2a70 0x7f6bac06b020 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7fb22cc73700 1 -- 172.21.15.83:0/2257243746 shutdown_connections 2023-11-28T11:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7effa6c6c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 0x7effa00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7effa6c6c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa0071c00 con 0x7effa006d370 2023-11-28T11:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7effa6c6c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa016d320 con 0x7effa016cf50 2023-11-28T11:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7effa6c6c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa016de80 con 0x7effa00899e0 2023-11-28T11:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89affd700 1 -- 172.21.15.83:0/1414703268 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890004230 con 0x7ff89404fb90 2023-11-28T11:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89affd700 1 -- 172.21.15.83:0/1414703268 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff890004390 con 0x7ff89404fb90 2023-11-28T11:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.609+0000 7ff89affd700 1 -- 172.21.15.83:0/1414703268 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8900036a0 con 0x7ff89404fb90 2023-11-28T11:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7eff9ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 0x7effa00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7eff9ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 0x7effa00710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:32982/0 (socket says 172.21.15.83:32982) 2023-11-28T11:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7eff9ffff700 1 -- 172.21.15.83:0/1757325690 learned_addr learned my addr 172.21.15.83:0/1757325690 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7ff8a11f0700 1 -- 172.21.15.83:0/1414703268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff89404fb90 msgr2=0x7ff894054470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7ff8a11f0700 1 --2- 172.21.15.83:0/1414703268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff89404fb90 0x7ff894054470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff890009870 tx=0x7ff890009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7ff8a11f0700 1 -- 172.21.15.83:0/1414703268 shutdown_connections 2023-11-28T11:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7ff8a11f0700 1 --2- 172.21.15.83:0/1414703268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff89404fb90 0x7ff894054470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7ff8a11f0700 1 -- 172.21.15.83:0/1414703268 >> 172.21.15.83:0/1414703268 conn(0x7ff8940515b0 msgr2=0x7ff894051980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7eff9ffff700 1 -- 172.21.15.83:0/1757325690 >> v1:172.21.15.90:6789/0 conn(0x7effa016cf50 legacy=0x7effa00c91d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7eff9ffff700 1 -- 172.21.15.83:0/1757325690 >> v1:172.21.15.83:6789/0 conn(0x7effa006d370 legacy=0x7effa006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== osd.1 v2:172.21.15.83:6809/103535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6bac0b3e20 con 0x7f6bac0b2a70 2023-11-28T11:08:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7eff9ffff700 1 -- 172.21.15.83:0/1757325690 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa016d5e0 con 0x7effa00899e0 2023-11-28T11:08:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7ff8a11f0700 1 -- 172.21.15.83:0/1414703268 shutdown_connections 2023-11-28T11:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.612+0000 7f0ec853a700 1 -- 172.21.15.83:0/4213284129 shutdown_connections 2023-11-28T11:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.611+0000 7ff8a11f0700 1 -- 172.21.15.83:0/1414703268 wait complete. 2023-11-28T11:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.612+0000 7ff8a11f0700 1 Processor -- start 2023-11-28T11:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.610+0000 7fb22cc73700 1 -- 172.21.15.83:0/2257243746 wait complete. 2023-11-28T11:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.612+0000 7fb22cc73700 1 Processor -- start 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.612+0000 7ff8a11f0700 1 -- start start 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7ff8a11f0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff89404fb90 0x7ff8940df200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7ff8a11f0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8940531e0 0x7ff8940d8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7ff8a11f0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8940d4630 0x7ff8940d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7ff8a11f0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940d7080 con 0x7ff89404fb90 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7ff8a11f0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940d7be0 con 0x7ff8940531e0 2023-11-28T11:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7ff8a11f0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89404e5b0 con 0x7ff8940d4630 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.612+0000 7fb22cc73700 1 -- start start 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb22cc73700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2280712a0 0x7fb228088330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb22cc73700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb228087c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb22cc73700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2280cc750 0x7fb228210a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb22cc73700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280881c0 con 0x7fb2280cc750 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb22cc73700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22820fd30 con 0x7fb2280712a0 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb22cc73700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22820ff70 con 0x7fb2280899e0 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7fb226d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2280cc750 0x7fb228210a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2280cc750 0x7fb228210a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48792/0 (socket says 172.21.15.83:48792) 2023-11-28T11:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 -- 172.21.15.83:0/566022741 learned_addr learned my addr 172.21.15.83:0/566022741 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.612+0000 7f0ec853a700 1 -- 172.21.15.83:0/4213284129 wait complete. 2023-11-28T11:08:54.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff8a09ef700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8940d4630 0x7ff8940d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7f0ec853a700 1 Processor -- start 2023-11-28T11:08:54.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7f0ec853a700 1 -- start start 2023-11-28T11:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff8a09ef700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8940d4630 0x7ff8940d85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:32988/0 (socket says 172.21.15.83:32988) 2023-11-28T11:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff8a09ef700 1 -- 172.21.15.83:0/105295271 learned_addr learned my addr 172.21.15.83:0/105295271 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff89bfff700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff89404fb90 0x7ff8940df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb225d9b700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb228087c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff89bfff700 1 -- 172.21.15.83:0/105295271 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8940d4630 msgr2=0x7ff8940d85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff89bfff700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8940d4630 0x7ff8940d85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.613+0000 7eff9ffff700 1 --2- 172.21.15.83:0/1757325690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 0x7effa00710c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff98009e80 tx=0x7eff9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16696430e8bc61ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7eff9f7fe700 1 -- 172.21.15.83:0/1757325690 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98003a20 con 0x7effa00899e0 2023-11-28T11:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7eff9f7fe700 1 -- 172.21.15.83:0/1757325690 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98003b80 con 0x7effa00899e0 2023-11-28T11:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7eff9f7fe700 1 -- 172.21.15.83:0/1757325690 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800b9c0 con 0x7effa00899e0 2023-11-28T11:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff89bfff700 1 -- 172.21.15.83:0/105295271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8940531e0 msgr2=0x7ff8940d8070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff89bfff700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8940531e0 0x7ff8940d8070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7ff89bfff700 1 -- 172.21.15.83:0/105295271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900094a0 con 0x7ff89404fb90 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7ff89bfff700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff89404fb90 0x7ff8940df200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff89c069a10 tx=0x7ff89c067e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 -- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 msgr2=0x7fb228087c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb2280899e0 0x7fb228087c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 -- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2280712a0 msgr2=0x7fb228088330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2280712a0 0x7fb228088330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.614+0000 7fb226d9d700 1 -- 172.21.15.83:0/566022741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009550 con 0x7fb2280cc750 2023-11-28T11:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7fb226d9d700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2280cc750 0x7fb228210a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb22000efc0 tx=0x7fb22000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec853a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec00b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec853a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ec0158520 0x7f0ec01480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec853a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ec015cd90 0x7f0ec0148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec853a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00ca380 con 0x7f0ec00cf290 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec853a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00b19f0 con 0x7f0ec015cd90 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec853a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00c6b40 con 0x7f0ec0158520 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec62d6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec00b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec62d6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec00b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48800/0 (socket says 172.21.15.83:48800) 2023-11-28T11:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec62d6700 1 -- 172.21.15.83:0/1653226652 learned_addr learned my addr 172.21.15.83:0/1653226652 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000cb20 con 0x7fb2280cc750 2023-11-28T11:08:54.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7effa6c6c700 1 -- 172.21.15.83:0/1757325690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 msgr2=0x7effa00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7effa6c6c700 1 --2- 172.21.15.83:0/1757325690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 0x7effa00710c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff98009e80 tx=0x7eff9800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:54.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7f0ec6ad7700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ec015cd90 0x7f0ec0148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c068670 con 0x7ff89404fb90 2023-11-28T11:08:54.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c072360 con 0x7ff89404fb90 2023-11-28T11:08:54.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c072710 con 0x7ff89404fb90 2023-11-28T11:08:54.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22000f040 con 0x7fb2280cc750 2023-11-28T11:08:54.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000cec0 con 0x7fb2280cc750 2023-11-28T11:08:54.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22816d5e0 con 0x7fb2280cc750 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.615+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb228071c00 con 0x7fb2280cc750 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7f0ec62d6700 1 -- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ec0158520 msgr2=0x7f0ec01480d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7f0ec62d6700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ec0158520 0x7f0ec01480d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7f0ec62d6700 1 -- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ec015cd90 msgr2=0x7f0ec0148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7f0ec62d6700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ec015cd90 0x7f0ec0148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7f0ec62d6700 1 -- 172.21.15.83:0/1653226652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc0094a0 con 0x7f0ec00cf290 2023-11-28T11:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7f0ec62d6700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec00b4280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc014040 tx=0x7f0ebc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89404f550 con 0x7ff89404fb90 2023-11-28T11:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7effa6c6c700 1 -- 172.21.15.83:0/1757325690 shutdown_connections 2023-11-28T11:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7effa6c6c700 1 --2- 172.21.15.83:0/1757325690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa00899e0 0x7effa00710c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7effa6c6c700 1 -- 172.21.15.83:0/1757325690 >> 172.21.15.83:0/1757325690 conn(0x7effa00b1380 msgr2=0x7effa00ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8940539c0 con 0x7ff89404fb90 2023-11-28T11:08:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.616+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb22816d8c0 con 0x7fb2280cc750 2023-11-28T11:08:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.617+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff894050090 con 0x7ff89404fb90 2023-11-28T11:08:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb220014480 con 0x7fb2280cc750 2023-11-28T11:08:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7fb216ffd700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb210068520 0x7fb21006a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7effa6c6c700 1 -- 172.21.15.83:0/1757325690 shutdown_connections 2023-11-28T11:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb220087050 con 0x7fb2280cc750 2023-11-28T11:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7fb216ffd700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fb2100721f0 0x7fb210074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 --> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb210074cf0 con 0x7fb2100721f0 2023-11-28T11:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb220087400 con 0x7fb2280cc750 2023-11-28T11:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.619+0000 7fb225d9b700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fb2100721f0 0x7fb210074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.618+0000 7effa6c6c700 1 -- 172.21.15.83:0/1757325690 wait complete. 2023-11-28T11:08:54.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.619+0000 7fb22659c700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb210068520 0x7fb21006a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff89c06b3f0 con 0x7ff89404fb90 2023-11-28T11:08:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7fb225d9b700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fb2100721f0 0x7fb210074620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7fb22659c700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb210068520 0x7fb21006a9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb2280c4840 tx=0x7fb218010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7ff898ff9700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff88c068840 0x7ff88c06acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff89c0ff770 con 0x7ff89404fb90 2023-11-28T11:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7ff898ff9700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7ff88c072510 0x7ff88c074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 --> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff88c075010 con 0x7ff88c072510 2023-11-28T11:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff89c0ffb20 con 0x7ff89404fb90 2023-11-28T11:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7effa6c6c700 1 Processor -- start 2023-11-28T11:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.620+0000 7effa6c6c700 1 -- start start 2023-11-28T11:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.621+0000 7ff8a09ef700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7ff88c072510 0x7ff88c074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.621+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== osd.4 v2:172.21.15.90:6816/103766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb210074cf0 con 0x7fb2100721f0 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa6c6c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa006d370 0x7effa00c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa6c6c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa00899e0 0x7effa01673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa6c6c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa016cf50 0x7effa00b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa6c6c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa0071600 con 0x7effa006d370 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa6c6c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c4f50 con 0x7effa00899e0 2023-11-28T11:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa6c6c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa016b990 con 0x7effa016cf50 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa4a08700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa006d370 0x7effa00c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa4a08700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa006d370 0x7effa00c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48816/0 (socket says 172.21.15.83:48816) 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7effa4a08700 1 -- 172.21.15.83:0/3469441458 learned_addr learned my addr 172.21.15.83:0/3469441458 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa4a08700 1 -- 172.21.15.83:0/3469441458 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa016cf50 msgr2=0x7effa00b19f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa4a08700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7effa016cf50 0x7effa00b19f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa4a08700 1 -- 172.21.15.83:0/3469441458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa00899e0 msgr2=0x7effa01673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa4a08700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa00899e0 0x7effa01673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa4a08700 1 -- 172.21.15.83:0/3469441458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009b30 con 0x7effa006d370 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa4a08700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa006d370 0x7effa00c4a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff9400b470 tx=0x7eff9400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94010970 con 0x7effa006d370 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94010af0 con 0x7effa006d370 2023-11-28T11:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94010ea0 con 0x7effa006d370 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa016dba0 con 0x7effa006d370 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.623+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa016d8c0 con 0x7effa006d370 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0036a0 con 0x7f0ec00cf290 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc01e070 con 0x7f0ec00cf290 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc003a40 con 0x7f0ec00cf290 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec0158b30 con 0x7f0ec00cf290 2023-11-28T11:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01589a0 con 0x7f0ec00cf290 2023-11-28T11:08:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.626+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ebc00f050 con 0x7f0ec00cf290 2023-11-28T11:08:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.626+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff9401f030 con 0x7effa006d370 2023-11-28T11:08:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.626+0000 7eff9d7fa700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff8c0682f0 0x7eff8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.626+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff94089630 con 0x7effa006d370 2023-11-28T11:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.627+0000 7f0eb2ffd700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea4068340 0x7f0ea406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.627+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ebc087100 con 0x7f0ec00cf290 2023-11-28T11:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.627+0000 7f0ec5ad5700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea4068340 0x7f0ea406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.621+0000 7ff8a09ef700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7ff88c072510 0x7ff88c074940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.622+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== osd.2 v2:172.21.15.83:6824/103544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff88c075010 con 0x7ff88c072510 2023-11-28T11:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.628+0000 7ff89b7fe700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff88c068840 0x7ff88c06acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.628+0000 7f0ec5ad5700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea4068340 0x7f0ea406a7f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ec01491b0 tx=0x7f0eb4007ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.628+0000 7ff89b7fe700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff88c068840 0x7ff88c06acf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8940de680 tx=0x7ff890008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7f0ec853a700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f0ea80015e0 0x7f0ea8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 --> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ea8006bc0 con 0x7f0ea80015e0 2023-11-28T11:08:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7effa6c6c700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7eff800015e0 0x7eff80003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 --> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff80006bc0 con 0x7eff800015e0 2023-11-28T11:08:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7f0ec6ad7700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f0ea80015e0 0x7f0ea8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7effa5209700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7eff800015e0 0x7eff80003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7f97ff08f700 1 Processor -- start 2023-11-28T11:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.630+0000 7f97ff08f700 1 -- start start 2023-11-28T11:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.631+0000 7fdaa25b3700 1 Processor -- start 2023-11-28T11:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97ff08f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8160cf0 0x7f97f80ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97ff08f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f8164b20 con 0x7f97f8070ad0 2023-11-28T11:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97ff08f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f8071530 con 0x7f97f80c1ab0 2023-11-28T11:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97ff08f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8070f70 con 0x7f97f8160cf0 2023-11-28T11:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.631+0000 7effa5209700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7eff800015e0 0x7eff80003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== osd.3 v2:172.21.15.83:6808/103549 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7eff80006bc0 con 0x7eff800015e0 2023-11-28T11:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7fdaa25b3700 1 -- start start 2023-11-28T11:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97fce2b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8160cf0 0x7f97f80ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97f7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f97f8070ad0 0x7f97f8164fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46688/0 (socket says 172.21.15.83:46688) 2023-11-28T11:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7f97f7fff700 1 -- 172.21.15.83:0/1139166021 learned_addr learned my addr 172.21.15.83:0/1139166021 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.632+0000 7eff9ffff700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff8c0682f0 0x7eff8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97f77fe700 1 -- 172.21.15.83:0/1139166021 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252500161 0 0) 0x7f97f8164b20 con 0x7f97f8070ad0 2023-11-28T11:08:54.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97f77fe700 1 -- 172.21.15.83:0/1139166021 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e00035b0 con 0x7f97f8070ad0 2023-11-28T11:08:54.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f0ec6ad7700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f0ea80015e0 0x7f0ea8003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97f77fe700 1 -- 172.21.15.83:0/1139166021 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835144120 0 0) 0x7f97e00035b0 con 0x7f97f8070ad0 2023-11-28T11:08:54.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97fce2b700 1 -- 172.21.15.83:0/1139166021 >> v1:172.21.15.90:6789/0 conn(0x7f97f80c1ab0 legacy=0x7f97f80c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97fce2b700 1 -- 172.21.15.83:0/1139166021 >> v1:172.21.15.83:6789/0 conn(0x7f97f8070ad0 legacy=0x7f97f8164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97fce2b700 1 -- 172.21.15.83:0/1139166021 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8071250 con 0x7f97f8160cf0 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fdaa25b3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fda9c06e440 0x7fda9c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fdaa25b3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda9c070ad0 0x7fda9c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fdaa25b3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c15d2d0 con 0x7fda9c158790 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fdaa25b3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c071490 con 0x7fda9c070ad0 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fdaa25b3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c06d7d0 con 0x7fda9c06e440 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fda9bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fda9c158790 0x7fda9c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46692/0 (socket says 172.21.15.83:46692) 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7fda9bfff700 1 -- 172.21.15.83:0/1783708698 learned_addr learned my addr 172.21.15.83:0/1783708698 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7eff9ffff700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff8c0682f0 0x7eff8c06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7effa00b4b60 tx=0x7eff98011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.634+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== osd.6 v2:172.21.15.90:6800/103764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0ea8006bc0 con 0x7f0ea80015e0 2023-11-28T11:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.633+0000 7f97fce2b700 1 --2- 172.21.15.83:0/1139166021 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8160cf0 0x7f97f80ae030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009870 tx=0x7f97ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee7d718f218565b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.636+0000 7f97f77fe700 1 -- 172.21.15.83:0/1139166021 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00019f0 con 0x7f97f8160cf0 2023-11-28T11:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.636+0000 7f97f77fe700 1 -- 172.21.15.83:0/1139166021 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec004030 con 0x7f97f8160cf0 2023-11-28T11:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.636+0000 7f97f77fe700 1 -- 172.21.15.83:0/1139166021 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0043c0 con 0x7f97f8160cf0 2023-11-28T11:08:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.637+0000 7f97ff08f700 1 -- 172.21.15.83:0/1139166021 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8160cf0 msgr2=0x7f97f80ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.637+0000 7f97ff08f700 1 --2- 172.21.15.83:0/1139166021 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8160cf0 0x7f97f80ae030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009870 tx=0x7f97ec009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085537812 0 0) 0x7fda8c001110 con 0x7fda9c158790 2023-11-28T11:08:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda840035b0 con 0x7fda9c158790 2023-11-28T11:08:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9b7fe700 1 --2- 172.21.15.83:0/1783708698 >> v2:172.21.15.90:3300/0 conn(0x7fda9c06e440 0x7fda9c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7f97ff08f700 1 -- 172.21.15.83:0/1139166021 shutdown_connections 2023-11-28T11:08:54.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7f97ff08f700 1 --2- 172.21.15.83:0/1139166021 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8160cf0 0x7f97f80ae030 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7f97ff08f700 1 -- 172.21.15.83:0/1139166021 >> 172.21.15.83:0/1139166021 conn(0x7f97f80b1380 msgr2=0x7f97f80ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2681694963 0 0) 0x7fda840035b0 con 0x7fda9c158790 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 >> v2:172.21.15.90:3300/0 conn(0x7fda9c06e440 msgr2=0x7fda9c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 --2- 172.21.15.83:0/1783708698 >> v2:172.21.15.90:3300/0 conn(0x7fda9c06e440 0x7fda9c15cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 >> v2:172.21.15.83:3300/0 conn(0x7fda9c070ad0 msgr2=0x7fda9c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 --2- 172.21.15.83:0/1783708698 >> v2:172.21.15.83:3300/0 conn(0x7fda9c070ad0 0x7fda9c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c06cea0 con 0x7fda9c158790 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fda8c001800 con 0x7fda9c158790 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 >> v1:172.21.15.83:6789/0 conn(0x7fda9c158790 legacy=0x7fda9c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda84004c30 0x7fda840037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda84003ff0 0x7fda84009060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda840095a0 0x7fda84009a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.639+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c071490 con 0x7fda840095a0 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c06d7d0 con 0x7fda84004c30 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8400a2c0 con 0x7fda84003ff0 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda840095a0 0x7fda84009a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 -- 172.21.15.83:0/1783708698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda84003ff0 msgr2=0x7fda84009060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda84003ff0 0x7fda84009060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 -- 172.21.15.83:0/1783708698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda84004c30 msgr2=0x7fda840037f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda84004c30 0x7fda840037f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 -- 172.21.15.83:0/1783708698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda840035b0 con 0x7fda840095a0 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 -- 172.21.15.83:0/1783708698 reap_dead start 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.640+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda840095a0 0x7fda84009a40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda94009e60 tx=0x7fda84004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ce3891328f29a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.641+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94003c00 con 0x7fda840095a0 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.641+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda94003d80 con 0x7fda840095a0 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.641+0000 7fda9affd700 1 -- 172.21.15.83:0/1783708698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400bae0 con 0x7fda840095a0 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 -- 172.21.15.83:0/1783708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda840095a0 msgr2=0x7fda84009a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda840095a0 0x7fda84009a40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda94009e60 tx=0x7fda84004870 comp rx=0 tx=0).stop 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 -- 172.21.15.83:0/1783708698 shutdown_connections 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 --2- 172.21.15.83:0/1783708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda840095a0 0x7fda84009a40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 -- 172.21.15.83:0/1783708698 >> 172.21.15.83:0/1783708698 conn(0x7fda9c0b1380 msgr2=0x7fda9c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 -- 172.21.15.83:0/1783708698 shutdown_connections 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.642+0000 7fdaa25b3700 1 -- 172.21.15.83:0/1783708698 wait complete. 2023-11-28T11:08:54.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.643+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 --> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6bac0ca330 con 0x7f6bac0b2a70 2023-11-28T11:08:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.644+0000 7fdaa25b3700 1 Processor -- start 2023-11-28T11:08:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.644+0000 7f6b9affd700 1 -- 172.21.15.83:0/1461010988 <== osd.1 v2:172.21.15.83:6809/103535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6bac0ca330 con 0x7f6bac0b2a70 2023-11-28T11:08:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.644+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 --> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2280742b0 con 0x7fb2100721f0 2023-11-28T11:08:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.644+0000 7fdaa25b3700 1 -- start start 2023-11-28T11:08:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7f97ff08f700 1 -- 172.21.15.83:0/1139166021 shutdown_connections 2023-11-28T11:08:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7f97ff08f700 1 -- 172.21.15.83:0/1139166021 wait complete. 2023-11-28T11:08:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fb216ffd700 1 -- 172.21.15.83:0/566022741 <== osd.4 v2:172.21.15.90:6816/103766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2280742b0 con 0x7fb2100721f0 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fdaa25b3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda840095a0 0x7fda9c0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fdaa25b3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c1f10a0 0x7fda9c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fdaa25b3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda9c1466b0 0x7fda9c146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fdaa25b3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c1f2090 con 0x7fda9c1f10a0 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fdaa25b3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c147040 con 0x7fda840095a0 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.645+0000 7fdaa25b3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c1f17f0 con 0x7fda9c1466b0 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c1f10a0 0x7fda9c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c1f10a0 0x7fda9c0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48842/0 (socket says 172.21.15.83:48842) 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 -- 172.21.15.83:0/2544583077 learned_addr learned my addr 172.21.15.83:0/2544583077 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda9c1466b0 0x7fda9c146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9bfff700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda840095a0 0x7fda9c0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 -- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda9c1466b0 msgr2=0x7fda9c146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fda9c1466b0 0x7fda9c146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 -- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda840095a0 msgr2=0x7fda9c0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda840095a0 0x7fda9c0b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fda9b7fe700 1 -- 172.21.15.83:0/2544583077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c06d460 con 0x7fda9c1f10a0 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7f97ff08f700 1 Processor -- start 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fb2100721f0 msgr2=0x7fb210074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7fb22cc73700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fb2100721f0 0x7fb210074620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.646+0000 7f97ff08f700 1 -- start start 2023-11-28T11:08:54.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7fda9b7fe700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c1f10a0 0x7fda9c0af230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0cf7d0 tx=0x7fda9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011800 con 0x7fda9c1f10a0 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90011960 con 0x7fda9c1f10a0 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011cf0 con 0x7fda9c1f10a0 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c06d180 con 0x7fda9c1f10a0 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f6bac0b2a70 msgr2=0x7f6bac06b020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7f6bb088c700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f6bac0b2a70 0x7f6bac06b020 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb210068520 msgr2=0x7fb21006a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb22cc73700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb210068520 0x7fb21006a9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb2280c4840 tx=0x7fb218010000 comp rx=0 tx=0).stop 2023-11-28T11:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2280cc750 msgr2=0x7fb228210a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb22cc73700 1 --2- 172.21.15.83:0/566022741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2280cc750 0x7fb228210a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb22000efc0 tx=0x7fb22000c0c0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb226d9d700 1 -- 172.21.15.83:0/566022741 reap_dead start 2023-11-28T11:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.647+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c158b60 con 0x7fda9c1f10a0 2023-11-28T11:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6b8c068340 msgr2=0x7f6b8c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6bb088c700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6b8c068340 0x7f6b8c06a7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0007af0 tx=0x7f6ba0008040 comp rx=0 tx=0).stop 2023-11-28T11:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bac15d3b0 msgr2=0x7f6bac158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6bb088c700 1 --2- 172.21.15.83:0/1461010988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bac15d3b0 0x7f6bac158d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c015000 tx=0x7f6b9c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda9c0742b0 con 0x7fda9c1f10a0 2023-11-28T11:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 shutdown_connections 2023-11-28T11:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 >> 172.21.15.83:0/566022741 conn(0x7fb2280b1380 msgr2=0x7fb22806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6baad9d700 1 -- 172.21.15.83:0/1461010988 reap_dead start 2023-11-28T11:08:54.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 shutdown_connections 2023-11-28T11:08:54.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 >> 172.21.15.83:0/1461010988 conn(0x7f6bac0b1090 msgr2=0x7f6bac15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97ff08f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8070ad0 0x7f97f80c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97ff08f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f80c1ab0 0x7f97f80c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97ff08f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f8160cf0 0x7f97f81f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97ff08f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f814e0d0 con 0x7f97f8160cf0 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97ff08f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c4d10 con 0x7f97f80c1ab0 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97ff08f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c4ff0 con 0x7f97f8070ad0 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f80c1ab0 0x7f97f80c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97fd62c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f8160cf0 0x7f97f81f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f80c1ab0 0x7f97f80c2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35842/0 (socket says 172.21.15.83:35842) 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97fd62c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f8160cf0 0x7f97f81f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48850/0 (socket says 172.21.15.83:48850) 2023-11-28T11:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 -- 172.21.15.83:0/3239414232 learned_addr learned my addr 172.21.15.83:0/3239414232 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 -- 172.21.15.83:0/3239414232 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8070ad0 msgr2=0x7f97f80c26b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97f8070ad0 0x7f97f80c26b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 -- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f8160cf0 msgr2=0x7f97f81f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97f8160cf0 0x7f97f81f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.648+0000 7f97f7fff700 1 -- 172.21.15.83:0/3239414232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0094a0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7f97f7fff700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f80c1ab0 0x7f97f80c2bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97f0001f70 tx=0x7f97f000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00020c0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000abe0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80cd1d0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00070d0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81647b0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda90017030 con 0x7fda9c1f10a0 2023-11-28T11:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.649+0000 7fda98ff9700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fda880682f0 0x7fda8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda90085930 con 0x7fda9c1f10a0 2023-11-28T11:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fda98ff9700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7fda88071f40 0x7fda88074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 --> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda88074a40 con 0x7fda88071f40 2023-11-28T11:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda90085cb0 con 0x7fda9c1f10a0 2023-11-28T11:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97d8000fc0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fda9bfff700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fda880682f0 0x7fda8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 shutdown_connections 2023-11-28T11:08:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fb22cc73700 1 -- 172.21.15.83:0/566022741 wait complete. 2023-11-28T11:08:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 shutdown_connections 2023-11-28T11:08:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7f6bb088c700 1 -- 172.21.15.83:0/1461010988 wait complete. 2023-11-28T11:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.650+0000 7fda9bfff700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fda880682f0 0x7fda8806a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0012b0 tx=0x7fda8c001be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.651+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7fda88071f40 0x7fda88074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.651+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97f002b030 con 0x7f97f80c1ab0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.651+0000 7f97f57fa700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.651+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97f0096cb0 con 0x7f97f80c1ab0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97f57fa700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7f97e4071f40 0x7f97e4074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 --> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97e4074a40 con 0x7f97e4071f40 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f009c390 con 0x7f97f80c1ab0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97fce2b700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97fd62c700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7f97e4071f40 0x7f97e4074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97fce2b700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009bc0 tx=0x7f97ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f97fd62c700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7f97e4071f40 0x7f97e4074370 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.653+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== osd.0 v2:172.21.15.83:6800/103534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f97e4074a40 con 0x7f97e4071f40 2023-11-28T11:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.654+0000 7fdaa0b50700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7fda88071f40 0x7fda88074370 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.651+0000 7f2a6649b700 1 Processor -- start 2023-11-28T11:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.651+0000 7f2a6649b700 1 -- start start 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f2a6649b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a600899e0 0x7f2a6016cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f2a6649b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2a600cc750 0x7f2a600ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f2a6649b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a60169860 con 0x7f2a60071330 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f2a6649b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6016de80 con 0x7f2a600899e0 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.652+0000 7f2a6649b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600cd110 con 0x7f2a600cc750 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.653+0000 7f2a5f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a600899e0 0x7f2a6016cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.653+0000 7f2a5f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a600899e0 0x7f2a6016cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48862/0 (socket says 172.21.15.83:48862) 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.653+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/1669906123 learned_addr learned my addr 172.21.15.83:0/1669906123 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.654+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/1669906123 >> v2:172.21.15.90:3300/0 conn(0x7f2a600cc750 msgr2=0x7f2a600ccbd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.654+0000 7f2a5f7fe700 1 --2- 172.21.15.83:0/1669906123 >> v2:172.21.15.90:3300/0 conn(0x7f2a600cc750 0x7f2a600ccbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.654+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/1669906123 >> v1:172.21.15.83:6789/0 conn(0x7f2a60071330 legacy=0x7f2a600c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.654+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/1669906123 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60071c00 con 0x7f2a600899e0 2023-11-28T11:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.654+0000 7f2a5f7fe700 1 --2- 172.21.15.83:0/1669906123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600899e0 0x7f2a6016cf50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce3d0 tx=0x7f2a54007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d78e51ec63a99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.655+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== osd.7 v2:172.21.15.90:6808/103765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fda88074a40 con 0x7fda88071f40 2023-11-28T11:08:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.655+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 --> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8940e24e0 con 0x7ff88c072510 2023-11-28T11:08:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff898ff9700 1 -- 172.21.15.83:0/105295271 <== osd.2 v2:172.21.15.83:6824/103544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8940e24e0 con 0x7ff88c072510 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7ff88c072510 msgr2=0x7ff88c074940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a11f0700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7ff88c072510 0x7ff88c074940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff88c068840 msgr2=0x7ff88c06acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a11f0700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff88c068840 0x7ff88c06acf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8940de680 tx=0x7ff890008030 comp rx=0 tx=0).stop 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff89404fb90 msgr2=0x7ff8940df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a11f0700 1 --2- 172.21.15.83:0/105295271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff89404fb90 0x7ff8940df200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff89c069a10 tx=0x7ff89c067e10 comp rx=0 tx=0).stop 2023-11-28T11:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.656+0000 7ff8a09ef700 1 -- 172.21.15.83:0/105295271 reap_dead start 2023-11-28T11:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.653+0000 7f2a5effd700 1 -- 172.21.15.83:0/1669906123 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992153716 0 0) 0x7f2a60169860 con 0x7f2a60071330 2023-11-28T11:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.657+0000 7f2a5effd700 1 -- 172.21.15.83:0/1669906123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400a880 con 0x7f2a600899e0 2023-11-28T11:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.657+0000 7f2a5effd700 1 -- 172.21.15.83:0/1669906123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400a9e0 con 0x7f2a600899e0 2023-11-28T11:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.657+0000 7f2a5effd700 1 -- 172.21.15.83:0/1669906123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54012480 con 0x7f2a600899e0 2023-11-28T11:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 shutdown_connections 2023-11-28T11:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 >> 172.21.15.83:0/105295271 conn(0x7ff8940515b0 msgr2=0x7ff894055000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 -- 172.21.15.83:0/1669906123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600899e0 msgr2=0x7f2a6016cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 --2- 172.21.15.83:0/1669906123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600899e0 0x7f2a6016cf50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce3d0 tx=0x7f2a54007bc0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 -- 172.21.15.83:0/1669906123 shutdown_connections 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 --2- 172.21.15.83:0/1669906123 >> v2:172.21.15.90:3300/0 conn(0x7f2a600cc750 0x7f2a600ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 --2- 172.21.15.83:0/1669906123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600899e0 0x7f2a6016cf50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 -- 172.21.15.83:0/1669906123 >> 172.21.15.83:0/1669906123 conn(0x7f2a600b1380 msgr2=0x7f2a600ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 -- 172.21.15.83:0/1669906123 shutdown_connections 2023-11-28T11:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.658+0000 7f2a6649b700 1 -- 172.21.15.83:0/1669906123 wait complete. 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.660+0000 7f2a6649b700 1 Processor -- start 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.660+0000 7f2a6649b700 1 -- start start 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a6649b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2a60071330 0x7f2a60207100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a6649b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a600899e0 0x7f2a600c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a6649b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600cc750 0x7f2a600817c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a6649b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c4a80 con 0x7f2a600cc750 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a6649b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c3e70 con 0x7f2a600899e0 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a6649b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c4150 con 0x7f2a60071330 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a5ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2a60071330 0x7f2a60207100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a5f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a600899e0 0x7f2a600c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a5f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a600899e0 0x7f2a600c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35862/0 (socket says 172.21.15.83:35862) 2023-11-28T11:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a5ffff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2a60071330 0x7f2a60207100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:33068/0 (socket says 172.21.15.83:33068) 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.662+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/605600268 learned_addr learned my addr 172.21.15.83:0/605600268 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2a60071330 msgr2=0x7f2a60207100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5f7fe700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2a60071330 0x7f2a60207100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600cc750 msgr2=0x7f2a600817c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5f7fe700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a600cc750 0x7f2a600817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5f7fe700 1 -- 172.21.15.83:0/605600268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540075c0 con 0x7f2a600899e0 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5f7fe700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a600899e0 0x7f2a600c4f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a600cf5e0 tx=0x7f2a54004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54004250 con 0x7f2a600899e0 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a540043b0 con 0x7f2a600899e0 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400ac90 con 0x7f2a600899e0 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6016d8c0 con 0x7f2a600899e0 2023-11-28T11:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6016d5e0 con 0x7f2a600899e0 2023-11-28T11:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.663+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 --> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff80005ca0 con 0x7eff800015e0 2023-11-28T11:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.665+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a5402c070 con 0x7f2a600899e0 2023-11-28T11:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.665+0000 7f2a5cff9700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2a4c0682f0 0x7f2a4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.665+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a54087f70 con 0x7f2a600899e0 2023-11-28T11:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.666+0000 7f2a5ffff700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2a4c0682f0 0x7f2a4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.666+0000 7f2a5ffff700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2a4c0682f0 0x7f2a4c06a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a50000f60 tx=0x7f2a50009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.690 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:54.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:08:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.670+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 --> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ea8005ca0 con 0x7f0ea80015e0 2023-11-28T11:08:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.670+0000 7eff9d7fa700 1 -- 172.21.15.83:0/3469441458 <== osd.3 v2:172.21.15.83:6808/103549 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7eff80005ca0 con 0x7eff800015e0 2023-11-28T11:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.670+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/1653226652 <== osd.6 v2:172.21.15.90:6800/103764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ea8005ca0 con 0x7f0ea80015e0 2023-11-28T11:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7eff800015e0 msgr2=0x7eff80003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa6c6c700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7eff800015e0 0x7eff80003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 shutdown_connections 2023-11-28T11:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff8c0682f0 msgr2=0x7eff8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa6c6c700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff8c0682f0 0x7eff8c06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7effa00b4b60 tx=0x7eff98011000 comp rx=0 tx=0).stop 2023-11-28T11:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa006d370 msgr2=0x7effa00c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa6c6c700 1 --2- 172.21.15.83:0/3469441458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa006d370 0x7effa00c4a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff9400b470 tx=0x7eff9400b830 comp rx=0 tx=0).stop 2023-11-28T11:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7effa5209700 1 -- 172.21.15.83:0/3469441458 reap_dead start 2023-11-28T11:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7ff8a11f0700 1 -- 172.21.15.83:0/105295271 wait complete. 2023-11-28T11:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f0ea80015e0 msgr2=0x7f0ea8003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f0ea80015e0 0x7f0ea8003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea4068340 msgr2=0x7f0ea406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea4068340 0x7f0ea406a7f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ec01491b0 tx=0x7f0eb4007ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 msgr2=0x7f0ec00b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 --2- 172.21.15.83:0/1653226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec00cf290 0x7f0ec00b4280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc014040 tx=0x7f0ebc004750 comp rx=0 tx=0).stop 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec6ad7700 1 -- 172.21.15.83:0/1653226652 reap_dead start 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 shutdown_connections 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 >> 172.21.15.83:0/1653226652 conn(0x7f0ec00b1380 msgr2=0x7f0ec00709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 shutdown_connections 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.671+0000 7f0ec853a700 1 -- 172.21.15.83:0/1653226652 wait complete. 2023-11-28T11:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.672+0000 7f2a6649b700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f2a400015e0 0x7f2a40003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.672+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a40006bc0 con 0x7f2a400015e0 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.672+0000 7f2a64a38700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f2a400015e0 0x7f2a40003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.672+0000 7f2a64a38700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f2a400015e0 0x7f2a40003a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.673+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 shutdown_connections 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.673+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 >> 172.21.15.83:0/3469441458 conn(0x7effa00b1380 msgr2=0x7effa00cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.673+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 shutdown_connections 2023-11-28T11:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.673+0000 7effa6c6c700 1 -- 172.21.15.83:0/3469441458 wait complete. 2023-11-28T11:08:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.673+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2a40006bc0 con 0x7f2a400015e0 2023-11-28T11:08:54.697 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:54.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:08:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.680+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 --> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97d8002d40 con 0x7f97e4071f40 2023-11-28T11:08:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97f57fa700 1 -- 172.21.15.83:0/3239414232 <== osd.0 v2:172.21.15.83:6800/103534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f97d8002d40 con 0x7f97e4071f40 2023-11-28T11:08:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7f97e4071f40 msgr2=0x7f97e4074370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97ff08f700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7f97e4071f40 0x7f97e4074370 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 msgr2=0x7f97e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97ff08f700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009bc0 tx=0x7f97ec008030 comp rx=0 tx=0).stop 2023-11-28T11:08:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f80c1ab0 msgr2=0x7f97f80c2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97ff08f700 1 --2- 172.21.15.83:0/3239414232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f80c1ab0 0x7f97f80c2bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97f0001f70 tx=0x7f97f000a340 comp rx=0 tx=0).stop 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.681+0000 7f97fd62c700 1 -- 172.21.15.83:0/3239414232 reap_dead start 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.682+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 shutdown_connections 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.682+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 >> 172.21.15.83:0/3239414232 conn(0x7f97f80b1380 msgr2=0x7f97f806cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.683+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 --> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda9c071b70 con 0x7fda88071f40 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.683+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 shutdown_connections 2023-11-28T11:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.683+0000 7f97ff08f700 1 -- 172.21.15.83:0/3239414232 wait complete. 2023-11-28T11:08:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fda98ff9700 1 -- 172.21.15.83:0/2544583077 <== osd.7 v2:172.21.15.90:6808/103765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda9c071b70 con 0x7fda88071f40 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7fda88071f40 msgr2=0x7fda88074370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7fda88071f40 0x7fda88074370 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fda880682f0 msgr2=0x7fda8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fda880682f0 0x7fda8806a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0012b0 tx=0x7fda8c001be0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c1f10a0 msgr2=0x7fda9c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 --2- 172.21.15.83:0/2544583077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c1f10a0 0x7fda9c0af230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0cf7d0 tx=0x7fda9000c8d0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa0b50700 1 -- 172.21.15.83:0/2544583077 reap_dead start 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 shutdown_connections 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.684+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 >> 172.21.15.83:0/2544583077 conn(0x7fda9c0b1380 msgr2=0x7fda9c06ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.685+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 shutdown_connections 2023-11-28T11:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.685+0000 7fdaa25b3700 1 -- 172.21.15.83:0/2544583077 wait complete. 2023-11-28T11:08:54.703 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:54.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:08:54.704 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2023-11-28T11:08:54.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:08:54.705 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:54.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:08:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.699+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a40005ca0 con 0x7f2a400015e0 2023-11-28T11:08:54.707 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:54.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:08:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a5cff9700 1 -- 172.21.15.83:0/605600268 <== osd.5 v2:172.21.15.90:6824/103767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2a40005ca0 con 0x7f2a400015e0 2023-11-28T11:08:54.708 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:54.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f2a400015e0 msgr2=0x7f2a40003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f2a400015e0 0x7f2a40003a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2a4c0682f0 msgr2=0x7f2a4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2a4c0682f0 0x7f2a4c06a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a50000f60 tx=0x7f2a50009040 comp rx=0 tx=0).stop 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a600899e0 msgr2=0x7f2a600c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 --2- 172.21.15.83:0/605600268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a600899e0 0x7f2a600c4f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a600cf5e0 tx=0x7f2a54004750 comp rx=0 tx=0).stop 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a64a38700 1 -- 172.21.15.83:0/605600268 reap_dead start 2023-11-28T11:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 shutdown_connections 2023-11-28T11:08:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 >> 172.21.15.83:0/605600268 conn(0x7f2a600b1380 msgr2=0x7f2a600ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 shutdown_connections 2023-11-28T11:08:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.700+0000 7f2a6649b700 1 -- 172.21.15.83:0/605600268 wait complete. 2023-11-28T11:08:54.715 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:54.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:08:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.881+0000 7fba82518700 1 Processor -- start 2023-11-28T11:08:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.881+0000 7fba82518700 1 -- start start 2023-11-28T11:08:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.882+0000 7fba82518700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fba7c071330 0x7fba7c16cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.882+0000 7fba82518700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0cc750 0x7fba7c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.882+0000 7fba82518700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7c169860 con 0x7fba7c0899e0 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.882+0000 7fba82518700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7c16de80 con 0x7fba7c071330 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.882+0000 7fba82518700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0cd110 con 0x7fba7c0cc750 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba80ab5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0cc750 0x7fba7c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7b7fe700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fba7c0899e0 0x7fba7c0c91d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45084/0 (socket says 172.21.15.83:45084) 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7b7fe700 1 -- 172.21.15.83:0/700260385 learned_addr learned my addr 172.21.15.83:0/700260385 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810058754 0 0) 0x7fba7c169860 con 0x7fba7c0899e0 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba640057c0 con 0x7fba7c0899e0 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7bfff700 1 --2- 172.21.15.83:0/700260385 >> v2:172.21.15.90:3300/0 conn(0x7fba7c071330 0x7fba7c16cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 <== mon.1 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1118847377 0 0) 0x7fba640057c0 con 0x7fba7c0899e0 2023-11-28T11:08:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0cc750 msgr2=0x7fba7c0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0cc750 0x7fba7c0ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 >> v2:172.21.15.90:3300/0 conn(0x7fba7c071330 msgr2=0x7fba7c16cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 --2- 172.21.15.83:0/700260385 >> v2:172.21.15.90:3300/0 conn(0x7fba7c071330 0x7fba7c16cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c071c00 con 0x7fba7c0899e0 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.883+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 <== mon.1 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fba740019f0 con 0x7fba7c0899e0 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 >> v1:172.21.15.90:6789/0 conn(0x7fba7c0899e0 legacy=0x7fba7c0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba64005c10 0x7fba64006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba64006640 0x7fba64006b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba64007120 0x7fba6400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c16de80 con 0x7fba64007120 2023-11-28T11:08:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0cd110 con 0x7fba64005c10 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6400e160 con 0x7fba64006640 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba64005c10 0x7fba64006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7b7fe700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba64006640 0x7fba64006b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 -- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba64006640 msgr2=0x7fba64006b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba64006640 0x7fba64006b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 -- 172.21.15.83:0/700260385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba64007120 msgr2=0x7fba6400dc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba64007120 0x7fba6400dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 -- 172.21.15.83:0/700260385 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c071c00 con 0x7fba64005c10 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7bfff700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba64005c10 0x7fba64006020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009b60 tx=0x7fba64007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b31cf5d3982d504 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.884+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00e6c0 con 0x7fba64005c10 2023-11-28T11:08:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.885+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c00e820 con 0x7fba64005c10 2023-11-28T11:08:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.885+0000 7fba7affd700 1 -- 172.21.15.83:0/700260385 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00eb90 con 0x7fba64005c10 2023-11-28T11:08:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.885+0000 7fba82518700 1 -- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba64005c10 msgr2=0x7fba64006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.885+0000 7fba82518700 1 --2- 172.21.15.83:0/700260385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba64005c10 0x7fba64006020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009b60 tx=0x7fba64007a90 comp rx=0 tx=0).stop 2023-11-28T11:08:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.887+0000 7fba80ab5700 1 -- 172.21.15.83:0/700260385 reap_dead start 2023-11-28T11:08:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.887+0000 7fba82518700 1 -- 172.21.15.83:0/700260385 shutdown_connections 2023-11-28T11:08:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.887+0000 7fba82518700 1 -- 172.21.15.83:0/700260385 >> 172.21.15.83:0/700260385 conn(0x7fba7c0b1380 msgr2=0x7fba7c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.893+0000 7fba82518700 1 -- 172.21.15.83:0/700260385 shutdown_connections 2023-11-28T11:08:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.893+0000 7fba82518700 1 -- 172.21.15.83:0/700260385 wait complete. 2023-11-28T11:08:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.895+0000 7fba82518700 1 Processor -- start 2023-11-28T11:08:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.896+0000 7fba82518700 1 -- start start 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba82518700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 0x7fba7c1f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba82518700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0c2b20 0x7fba7c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba82518700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c084420 0x7fba7c1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba82518700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1f6910 con 0x7fba7c0c2750 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba82518700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0c53d0 con 0x7fba7c084420 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba82518700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0c5630 con 0x7fba7c0c2b20 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba7bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 0x7fba7c1f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba7bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 0x7fba7c1f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48886/0 (socket says 172.21.15.83:48886) 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba7bfff700 1 -- 172.21.15.83:0/2303686650 learned_addr learned my addr 172.21.15.83:0/2303686650 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.897+0000 7fba80ab5700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c084420 0x7fba7c1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7b7fe700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0c2b20 0x7fba7c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7bfff700 1 -- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0c2b20 msgr2=0x7fba7c0c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7bfff700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0c2b20 0x7fba7c0c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7bfff700 1 -- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c084420 msgr2=0x7fba7c1f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7bfff700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c084420 0x7fba7c1f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7bfff700 1 -- 172.21.15.83:0/2303686650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c16d8c0 con 0x7fba7c0c2750 2023-11-28T11:08:54.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.898+0000 7fba7bfff700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 0x7fba7c1f90f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba6c016000 tx=0x7fba6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.905+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00e6c0 con 0x7fba7c0c2750 2023-11-28T11:08:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.905+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c0038c0 con 0x7fba7c0c2750 2023-11-28T11:08:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.905+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c003c70 con 0x7fba7c0c2750 2023-11-28T11:08:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.906+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c16d5e0 con 0x7fba7c0c2750 2023-11-28T11:08:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.906+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c16dba0 con 0x7fba7c0c2750 2023-11-28T11:08:54.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.909+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba6c025070 con 0x7fba7c0c2750 2023-11-28T11:08:54.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.910+0000 7fba78ff9700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fba60068310 0x7fba6006a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.910+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fba6c087470 con 0x7fba7c0c2750 2023-11-28T11:08:54.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.910+0000 7fba7b7fe700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fba60068310 0x7fba6006a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.910+0000 7fba7b7fe700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fba60068310 0x7fba6006a7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba74004c50 tx=0x7fba74006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.914+0000 7faa16372700 1 Processor -- start 2023-11-28T11:08:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.914+0000 7faa16372700 1 -- start start 2023-11-28T11:08:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa16372700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa1006e440 0x7faa1015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa16372700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa10070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa16372700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1015d2d0 con 0x7faa10158790 2023-11-28T11:08:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa16372700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa10071490 con 0x7faa1006e440 2023-11-28T11:08:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa16372700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1006d7d0 con 0x7faa10070ad0 2023-11-28T11:08:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa1490f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa10070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.915+0000 7faa0ffff700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7faa10158790 0x7faa1006dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45100/0 (socket says 172.21.15.83:45100) 2023-11-28T11:08:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.916+0000 7faa0ffff700 1 -- 172.21.15.83:0/218136017 learned_addr learned my addr 172.21.15.83:0/218136017 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.916+0000 7faa1490f700 1 -- 172.21.15.83:0/218136017 >> v2:172.21.15.83:3300/0 conn(0x7faa1006e440 msgr2=0x7faa1015cd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.916+0000 7faa1490f700 1 --2- 172.21.15.83:0/218136017 >> v2:172.21.15.83:3300/0 conn(0x7faa1006e440 0x7faa1015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.916+0000 7faa1490f700 1 -- 172.21.15.83:0/218136017 >> v1:172.21.15.90:6789/0 conn(0x7faa10158790 legacy=0x7faa1006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.916+0000 7faa1490f700 1 -- 172.21.15.83:0/218136017 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1006cea0 con 0x7faa10070ad0 2023-11-28T11:08:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.916+0000 7faa1490f700 1 --2- 172.21.15.83:0/218136017 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa10070f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faa0800be30 tx=0x7faa08010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28648bf3a0aebb9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.921+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba580052f0 con 0x7fba7c0c2750 2023-11-28T11:08:54.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.924+0000 7faa0effd700 1 -- 172.21.15.83:0/218136017 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800b2d0 con 0x7faa10070ad0 2023-11-28T11:08:54.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.924+0000 7faa0effd700 1 -- 172.21.15.83:0/218136017 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa08009540 con 0x7faa10070ad0 2023-11-28T11:08:54.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.924+0000 7faa0effd700 1 -- 172.21.15.83:0/218136017 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080098f0 con 0x7faa10070ad0 2023-11-28T11:08:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.926+0000 7faa16372700 1 -- 172.21.15.83:0/218136017 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 msgr2=0x7faa10070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.926+0000 7faa16372700 1 --2- 172.21.15.83:0/218136017 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa10070f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faa0800be30 tx=0x7faa08010730 comp rx=0 tx=0).stop 2023-11-28T11:08:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.926+0000 7faa16372700 1 -- 172.21.15.83:0/218136017 shutdown_connections 2023-11-28T11:08:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.926+0000 7faa16372700 1 --2- 172.21.15.83:0/218136017 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa10070f50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.926+0000 7faa16372700 1 --2- 172.21.15.83:0/218136017 >> v2:172.21.15.83:3300/0 conn(0x7faa1006e440 0x7faa1015cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.926+0000 7faa16372700 1 -- 172.21.15.83:0/218136017 >> 172.21.15.83:0/218136017 conn(0x7faa100b1380 msgr2=0x7faa100ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.929+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba6c053a80 con 0x7fba7c0c2750 2023-11-28T11:08:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.929+0000 7faa16372700 1 -- 172.21.15.83:0/218136017 shutdown_connections 2023-11-28T11:08:54.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.933+0000 7faa16372700 1 -- 172.21.15.83:0/218136017 wait complete. 2023-11-28T11:08:54.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.934+0000 7faa16372700 1 Processor -- start 2023-11-28T11:08:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.935+0000 7faa16372700 1 -- start start 2023-11-28T11:08:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa16372700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa1006e440 0x7faa100b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa16372700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa100af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa16372700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 0x7faa100b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa16372700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100b30d0 con 0x7faa10158790 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa16372700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100b2d30 con 0x7faa1006e440 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa16372700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa100b2240 con 0x7faa10070ad0 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa1490f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 0x7faa100b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa0ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa1006e440 0x7faa100b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa1490f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 0x7faa100b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48904/0 (socket says 172.21.15.83:48904) 2023-11-28T11:08:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.936+0000 7faa1490f700 1 -- 172.21.15.83:0/1828739153 learned_addr learned my addr 172.21.15.83:0/1828739153 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa0f7fe700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa100af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa1490f700 1 -- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 msgr2=0x7faa100af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa1490f700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa100af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa1490f700 1 -- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa1006e440 msgr2=0x7faa100b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa1490f700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa1006e440 0x7faa100b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa1490f700 1 -- 172.21.15.83:0/1828739153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0800ba90 con 0x7faa10158790 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa0ffff700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa1006e440 0x7faa100b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.937+0000 7faa1490f700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 0x7faa100b34f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa100715d0 tx=0x7faa08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.938+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08010070 con 0x7faa10158790 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.938+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800ab50 con 0x7faa10158790 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.938+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08010290 con 0x7faa10158790 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.938+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1006d460 con 0x7faa10158790 2023-11-28T11:08:54.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.938+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1006d180 con 0x7faa10158790 2023-11-28T11:08:54.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.940+0000 7f62d6f15700 1 Processor -- start 2023-11-28T11:08:54.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.940+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa08028030 con 0x7faa10158790 2023-11-28T11:08:54.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.941+0000 7faa0cff9700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa9fc0683c0 0x7fa9fc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.941+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faa08087240 con 0x7faa10158790 2023-11-28T11:08:54.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.941+0000 7faa0ffff700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa9fc0683c0 0x7fa9fc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.942+0000 7faa0ffff700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa9fc0683c0 0x7fa9fc06a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa000050b0 tx=0x7faa00004f20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.943+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa10158b60 con 0x7faa10158790 2023-11-28T11:08:54.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.944+0000 7f62d6f15700 1 -- start start 2023-11-28T11:08:54.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.945+0000 7f62d6f15700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62d00cf290 0x7f62d006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.945+0000 7f62d6f15700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f62d0158520 0x7f62d0070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.945+0000 7f62d6f15700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d006d7d0 con 0x7f62d015cd90 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.945+0000 7f62d6f15700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d006d180 con 0x7f62d00cf290 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.945+0000 7f62d6f15700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d006d460 con 0x7f62d0158520 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62d00cf290 0x7f62d006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62d00cf290 0x7f62d006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48908/0 (socket says 172.21.15.83:48908) 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 -- 172.21.15.83:0/264232957 learned_addr learned my addr 172.21.15.83:0/264232957 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 -- 172.21.15.83:0/264232957 >> v2:172.21.15.90:3300/0 conn(0x7f62d0158520 msgr2=0x7f62d0070ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 --2- 172.21.15.83:0/264232957 >> v2:172.21.15.90:3300/0 conn(0x7f62d0158520 0x7f62d0070ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 -- 172.21.15.83:0/264232957 >> v1:172.21.15.90:6789/0 conn(0x7f62d015cd90 legacy=0x7f62d006dc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.947+0000 7f62d4cb1700 1 -- 172.21.15.83:0/264232957 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d015d1e0 con 0x7f62d00cf290 2023-11-28T11:08:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.948+0000 7f62d4cb1700 1 --2- 172.21.15.83:0/264232957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d00cf290 0x7f62d006e440 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f62d00ae5a0 tx=0x7f62c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f285beb68b4d8a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.950+0000 7f62cf7fe700 1 -- 172.21.15.83:0/264232957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80092d0 con 0x7f62d00cf290 2023-11-28T11:08:54.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.950+0000 7f62cf7fe700 1 -- 172.21.15.83:0/264232957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c801f070 con 0x7f62d00cf290 2023-11-28T11:08:54.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.950+0000 7f62cf7fe700 1 -- 172.21.15.83:0/264232957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8003c60 con 0x7f62d00cf290 2023-11-28T11:08:54.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.950+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa08053740 con 0x7faa10158790 2023-11-28T11:08:54.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.951+0000 7f7009050700 1 Processor -- start 2023-11-28T11:08:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.951+0000 7f62d6f15700 1 -- 172.21.15.83:0/264232957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d00cf290 msgr2=0x7f62d006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.951+0000 7f62d6f15700 1 --2- 172.21.15.83:0/264232957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d00cf290 0x7f62d006e440 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f62d00ae5a0 tx=0x7f62c800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f62d6f15700 1 -- 172.21.15.83:0/264232957 shutdown_connections 2023-11-28T11:08:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f62d6f15700 1 --2- 172.21.15.83:0/264232957 >> v2:172.21.15.90:3300/0 conn(0x7f62d0158520 0x7f62d0070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f62d6f15700 1 --2- 172.21.15.83:0/264232957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d00cf290 0x7f62d006e440 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f62d6f15700 1 -- 172.21.15.83:0/264232957 >> 172.21.15.83:0/264232957 conn(0x7f62d00b1380 msgr2=0x7f62d00ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f62d6f15700 1 -- 172.21.15.83:0/264232957 shutdown_connections 2023-11-28T11:08:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f62d6f15700 1 -- 172.21.15.83:0/264232957 wait complete. 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.951+0000 7f7009050700 1 -- start start 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f7009050700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004158790 0x7f700406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f7009050700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f700415cd90 0x7f700406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f7009050700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700406d7d0 con 0x7f7004070ad0 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f7009050700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700406e1a0 con 0x7f700415cd90 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f7009050700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700406cea0 con 0x7f7004158790 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f700359e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7004070ad0 0x7f70040c91d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46772/0 (socket says 172.21.15.83:46772) 2023-11-28T11:08:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.952+0000 7f700359e700 1 -- 172.21.15.83:0/1655918918 learned_addr learned my addr 172.21.15.83:0/1655918918 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412762817 0 0) 0x7f700406d7d0 con 0x7f7004070ad0 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec003e40 con 0x7f7004070ad0 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f700259c700 1 --2- 172.21.15.83:0/1655918918 >> v2:172.21.15.83:3300/0 conn(0x7f700415cd90 0x7f700406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1959637324 0 0) 0x7f6fec003e40 con 0x7f7004070ad0 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004158790 msgr2=0x7f700406e440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004158790 0x7f700406e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 >> v2:172.21.15.83:3300/0 conn(0x7f700415cd90 msgr2=0x7f700406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 --2- 172.21.15.83:0/1655918918 >> v2:172.21.15.83:3300/0 conn(0x7f700415cd90 0x7f700406dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700406d180 con 0x7f7004070ad0 2023-11-28T11:08:54.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f62d6f15700 1 Processor -- start 2023-11-28T11:08:54.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6ff4001670 con 0x7f7004070ad0 2023-11-28T11:08:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 >> v1:172.21.15.83:6789/0 conn(0x7f7004070ad0 legacy=0x7f70040c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fec0036e0 0x7f6fec004080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fec0045c0 0x7f6fec004a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fec0051e0 0x7f6fec005650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700406e1a0 con 0x7f6fec0051e0 2023-11-28T11:08:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700406cea0 con 0x7f6fec0045c0 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec005fb0 con 0x7f6fec0036e0 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.953+0000 7f700359e700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fec0051e0 0x7f6fec005650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f700359e700 1 -- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fec0036e0 msgr2=0x7f6fec004080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f700359e700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fec0036e0 0x7f6fec004080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f700359e700 1 -- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fec0045c0 msgr2=0x7f6fec004a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f700359e700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fec0045c0 0x7f6fec004a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f700359e700 1 -- 172.21.15.83:0/1655918918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec003e40 con 0x7f6fec0051e0 2023-11-28T11:08:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f700359e700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fec0051e0 0x7f6fec005650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c270 tx=0x7f6fec006110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f65923c6d33999f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400cd70 con 0x7f6fec0051e0 2023-11-28T11:08:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4005960 con 0x7f6fec0051e0 2023-11-28T11:08:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f7001d9b700 1 -- 172.21.15.83:0/1655918918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4005d10 con 0x7f6fec0051e0 2023-11-28T11:08:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f7009050700 1 -- 172.21.15.83:0/1655918918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fec0051e0 msgr2=0x7f6fec005650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f7009050700 1 --2- 172.21.15.83:0/1655918918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fec0051e0 0x7f6fec005650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c270 tx=0x7f6fec006110 comp rx=0 tx=0).stop 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.954+0000 7f62d6f15700 1 -- start start 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d6f15700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 0x7f62d00c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d6f15700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d0158520 0x7f62d00c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d6f15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d015cd90 0x7f62d00c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d6f15700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b3360 con 0x7f62d015cd90 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d6f15700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b2ed0 con 0x7f62d0158520 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d6f15700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b25a0 con 0x7f62d00cf290 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d4cb1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 0x7f62d00c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d4cb1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 0x7f62d00c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:33120/0 (socket says 172.21.15.83:33120) 2023-11-28T11:08:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d4cb1700 1 -- 172.21.15.83:0/112695953 learned_addr learned my addr 172.21.15.83:0/112695953 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62d54b2700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d015cd90 0x7f62d00c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.955+0000 7f62cffff700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d0158520 0x7f62d00c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f700359e700 1 -- 172.21.15.83:0/1655918918 reap_dead start 2023-11-28T11:08:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d4cb1700 1 -- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d0158520 msgr2=0x7f62d00c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d4cb1700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d0158520 0x7f62d00c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d4cb1700 1 -- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d015cd90 msgr2=0x7f62d00c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d4cb1700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d015cd90 0x7f62d00c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d4cb1700 1 -- 172.21.15.83:0/112695953 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c8009b30 con 0x7f62d00cf290 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f7009050700 1 -- 172.21.15.83:0/1655918918 shutdown_connections 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f7009050700 1 -- 172.21.15.83:0/1655918918 >> 172.21.15.83:0/1655918918 conn(0x7f70040b1380 msgr2=0x7f70040ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d4cb1700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 0x7f62d00c4880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62c8026040 tx=0x7f62c8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f7009050700 1 -- 172.21.15.83:0/1655918918 shutdown_connections 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800bf40 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d0158b30 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.956+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d01589a0 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.957+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80094f0 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.957+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8004270 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.958+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b00052f0 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.958+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62c803e070 con 0x7f62d00cf290 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.959+0000 7f62cd7fa700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62bc0682f0 0x7f62bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.959+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f62c8098250 con 0x7f62d00cf290 2023-11-28T11:08:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.960+0000 7fa56bdd2700 1 Processor -- start 2023-11-28T11:08:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7f62cffff700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62bc0682f0 0x7f62bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.959+0000 7f7009050700 1 -- 172.21.15.83:0/1655918918 wait complete. 2023-11-28T11:08:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7f7009050700 1 Processor -- start 2023-11-28T11:08:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.960+0000 7fa56bdd2700 1 -- start start 2023-11-28T11:08:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7fa56bdd2700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa564160cd0 0x7fa5640cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7fa56bdd2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5640c21c0 0x7fa56406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7fa56bdd2700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640c91d0 con 0x7fa5640cce20 2023-11-28T11:08:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7fa56bdd2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5641649f0 con 0x7fa5640c21c0 2023-11-28T11:08:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7fa56bdd2700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa564164cd0 con 0x7fa564160cd0 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5640c21c0 0x7fa56406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5640c21c0 0x7fa56406d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48928/0 (socket says 172.21.15.83:48928) 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 -- 172.21.15.83:0/3686695598 learned_addr learned my addr 172.21.15.83:0/3686695598 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 -- 172.21.15.83:0/3686695598 >> v2:172.21.15.90:3300/0 conn(0x7fa564160cd0 msgr2=0x7fa5640cc8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 --2- 172.21.15.83:0/3686695598 >> v2:172.21.15.90:3300/0 conn(0x7fa564160cd0 0x7fa5640cc8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 -- 172.21.15.83:0/3686695598 >> v1:172.21.15.90:6789/0 conn(0x7fa5640cce20 legacy=0x7fa5640c1ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7fa56a36f700 1 -- 172.21.15.83:0/3686695598 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564164fb0 con 0x7fa5640c21c0 2023-11-28T11:08:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7f7009050700 1 -- start start 2023-11-28T11:08:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f7009050700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004145060 0x7f70040b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f7009050700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70040b3120 0x7f70040b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f7009050700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004145430 0x7f70040b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f7009050700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70041554c0 con 0x7f70040b3120 2023-11-28T11:08:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f7009050700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70040b2080 con 0x7f7004145060 2023-11-28T11:08:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f7009050700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040b2360 con 0x7f7004145430 2023-11-28T11:08:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f700259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70040b3120 0x7f70040b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f700259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70040b3120 0x7f70040b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48930/0 (socket says 172.21.15.83:48930) 2023-11-28T11:08:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f700259c700 1 -- 172.21.15.83:0/2689899935 learned_addr learned my addr 172.21.15.83:0/2689899935 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f700359e700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004145430 0x7f70040b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7f62cffff700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62bc0682f0 0x7f62bc06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62d00c7870 tx=0x7f62c0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7fa56a36f700 1 --2- 172.21.15.83:0/3686695598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640c21c0 0x7fa56406d090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5640708d0 tx=0x7fa55c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf8baae3d706202 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f7002d9d700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004145060 0x7f70040b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7fa568b6c700 1 -- 172.21.15.83:0/3686695598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00e070 con 0x7fa5640c21c0 2023-11-28T11:08:54.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f700359e700 1 -- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004145060 msgr2=0x7f70040b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f700359e700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004145060 0x7f70040b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f700359e700 1 -- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70040b3120 msgr2=0x7f70040b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f700359e700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70040b3120 0x7f70040b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f700359e700 1 -- 172.21.15.83:0/2689899935 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700415d1e0 con 0x7f7004145430 2023-11-28T11:08:54.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.961+0000 7f9001c78700 1 Processor -- start 2023-11-28T11:08:54.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.962+0000 7f9001c78700 1 -- start start 2023-11-28T11:08:54.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f9001c78700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c21c0 0x7f8ffc0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f9001c78700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ffc164b20 con 0x7f8ffc070ad0 2023-11-28T11:08:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f9001c78700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ffc071530 con 0x7f8ffc160cf0 2023-11-28T11:08:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f9001c78700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc070f70 con 0x7f8ffc0c21c0 2023-11-28T11:08:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f8ffb7fe700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f8ffc160cf0 0x7f8ffc164fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45144/0 (socket says 172.21.15.83:45144) 2023-11-28T11:08:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f8ffb7fe700 1 -- 172.21.15.83:0/3306379071 learned_addr learned my addr 172.21.15.83:0/3306379071 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7f8ffbfff700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c21c0 0x7f8ffc0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.963+0000 7fa568b6c700 1 -- 172.21.15.83:0/3686695598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c003a20 con 0x7fa5640c21c0 2023-11-28T11:08:54.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa568b6c700 1 -- 172.21.15.83:0/3686695598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c003dd0 con 0x7fa5640c21c0 2023-11-28T11:08:54.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa56bdd2700 1 -- 172.21.15.83:0/3686695598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640c21c0 msgr2=0x7fa56406d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/3686695598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640c21c0 0x7fa56406d090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5640708d0 tx=0x7fa55c00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:54.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946575212 0 0) 0x7f8ffc164b20 con 0x7f8ffc070ad0 2023-11-28T11:08:54.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe40035b0 con 0x7f8ffc070ad0 2023-11-28T11:08:54.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7f700359e700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004145430 0x7f70040b4470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c240 tx=0x7f6ff4001930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62c80647c0 con 0x7f62d00cf290 2023-11-28T11:08:54.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa56bdd2700 1 -- 172.21.15.83:0/3686695598 shutdown_connections 2023-11-28T11:08:54.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/3686695598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640c21c0 0x7fa56406d090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/3686695598 >> v2:172.21.15.90:3300/0 conn(0x7fa564160cd0 0x7fa5640cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7fa56bdd2700 1 -- 172.21.15.83:0/3686695598 >> 172.21.15.83:0/3686695598 conn(0x7fa5640ceb80 msgr2=0x7fa5640cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029421688 0 0) 0x7f8ffc071530 con 0x7f8ffc160cf0 2023-11-28T11:08:54.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.964+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ffc164b20 con 0x7f8ffc160cf0 2023-11-28T11:08:54.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.1 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144827960 0 0) 0x7f8ffc164b20 con 0x7f8ffc160cf0 2023-11-28T11:08:54.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c21c0 msgr2=0x7f8ffc0ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c21c0 0x7f8ffc0ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 >> v1:172.21.15.83:6789/0 conn(0x7f8ffc070ad0 legacy=0x7f8ffc0c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc071250 con 0x7f8ffc160cf0 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.1 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f8fec001670 con 0x7f8ffc160cf0 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 >> v1:172.21.15.90:6789/0 conn(0x7f8ffc160cf0 legacy=0x7f8ffc164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fe4004bf0 0x7f8fe4004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8fe40037f0 0x7f8fe4003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8fe400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc070f70 con 0x7f8fe40041a0 2023-11-28T11:08:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe400c0e0 con 0x7f8fe4004bf0 2023-11-28T11:08:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe400c2a0 con 0x7f8fe40037f0 2023-11-28T11:08:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffb7fe700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fe4004bf0 0x7f8fe4004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffaffd700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8fe40037f0 0x7f8fe4003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4006e10 con 0x7f7004145430 2023-11-28T11:08:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4001aa0 con 0x7f7004145430 2023-11-28T11:08:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4007520 con 0x7f7004145430 2023-11-28T11:08:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700406d460 con 0x7f7004145430 2023-11-28T11:08:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.965+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7004158b60 con 0x7f7004145430 2023-11-28T11:08:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffbfff700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8fe400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffbfff700 1 -- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8fe40037f0 msgr2=0x7f8fe4003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffbfff700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8fe40037f0 0x7f8fe4003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffbfff700 1 -- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fe4004bf0 msgr2=0x7f8fe4004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffbfff700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fe4004bf0 0x7f8fe4004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.966+0000 7f8ffbfff700 1 -- 172.21.15.83:0/3306379071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc071250 con 0x7f8fe40041a0 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.967+0000 7f8ffbfff700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8fe400ba10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0c5be0 tx=0x7f8fe4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972cc1cf9a1f9b17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.967+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400eb20 con 0x7f8fe40041a0 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.967+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400f040 con 0x7f8fe40041a0 2023-11-28T11:08:54.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.967+0000 7f8ffa7fc700 1 -- 172.21.15.83:0/3306379071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400eea0 con 0x7f8fe40041a0 2023-11-28T11:08:54.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f9001c78700 1 -- 172.21.15.83:0/3306379071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 msgr2=0x7f8fe400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f9001c78700 1 --2- 172.21.15.83:0/3306379071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8fe400ba10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0c5be0 tx=0x7f8fe4005880 comp rx=0 tx=0).stop 2023-11-28T11:08:54.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f8ffbfff700 1 -- 172.21.15.83:0/3306379071 reap_dead start 2023-11-28T11:08:54.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f9001c78700 1 -- 172.21.15.83:0/3306379071 shutdown_connections 2023-11-28T11:08:54.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f9001c78700 1 -- 172.21.15.83:0/3306379071 >> 172.21.15.83:0/3306379071 conn(0x7f8ffc0b1380 msgr2=0x7f8ffc0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f9001c78700 1 -- 172.21.15.83:0/3306379071 shutdown_connections 2023-11-28T11:08:54.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f9001c78700 1 -- 172.21.15.83:0/3306379071 wait complete. 2023-11-28T11:08:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.970+0000 7f9001c78700 1 Processor -- start 2023-11-28T11:08:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.970+0000 7f9001c78700 1 -- start start 2023-11-28T11:08:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f9001c78700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8ffc0c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f9001c78700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc0c2bf0 0x7f8ffc0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f9001c78700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c50e0 0x7f8ffc0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f9001c78700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0c5560 con 0x7f8fe40041a0 2023-11-28T11:08:54.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f9001c78700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc144bf0 con 0x7f8ffc0c2bf0 2023-11-28T11:08:54.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f9001c78700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0c3100 con 0x7f8ffc0c50e0 2023-11-28T11:08:54.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f8ffb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8ffc0c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f8ffb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8ffc0c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48954/0 (socket says 172.21.15.83:48954) 2023-11-28T11:08:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.971+0000 7f8ffb7fe700 1 -- 172.21.15.83:0/3716561908 learned_addr learned my addr 172.21.15.83:0/3716561908 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffbfff700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c50e0 0x7f8ffc0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffaffd700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc0c2bf0 0x7f8ffc0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffb7fe700 1 -- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c50e0 msgr2=0x7f8ffc0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffb7fe700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c50e0 0x7f8ffc0b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffb7fe700 1 -- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc0c2bf0 msgr2=0x7f8ffc0c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffb7fe700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc0c2bf0 0x7f8ffc0c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffb7fe700 1 -- 172.21.15.83:0/3716561908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0cd1d0 con 0x7f8fe40041a0 2023-11-28T11:08:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8ffb7fe700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8ffc0c26b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0017a0 tx=0x7f8fec0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00f6e0 con 0x7f8fe40041a0 2023-11-28T11:08:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec00f880 con 0x7f8fe40041a0 2023-11-28T11:08:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1647b0 con 0x7f8fe40041a0 2023-11-28T11:08:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.972+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc0add10 con 0x7f8fe40041a0 2023-11-28T11:08:54.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f2c61e51700 1 Processor -- start 2023-11-28T11:08:54.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.968+0000 7f2c61e51700 1 -- start start 2023-11-28T11:08:54.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c61e51700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2c5c071330 0x7f2c5c16cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c61e51700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c5c0cc750 0x7f2c5c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c61e51700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c169860 con 0x7f2c5c0899e0 2023-11-28T11:08:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c61e51700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c16de80 con 0x7f2c5c0cc750 2023-11-28T11:08:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c61e51700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0cd110 con 0x7f2c5c071330 2023-11-28T11:08:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c5bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c5c0cc750 0x7f2c5c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c5bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c5c0cc750 0x7f2c5c0ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48952/0 (socket says 172.21.15.83:48952) 2023-11-28T11:08:54.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c5bfff700 1 -- 172.21.15.83:0/3282274056 learned_addr learned my addr 172.21.15.83:0/3282274056 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:54.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.970+0000 7f2c5a7fc700 1 -- 172.21.15.83:0/3282274056 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007607913 0 0) 0x7f2c5c169860 con 0x7f2c5c0899e0 2023-11-28T11:08:54.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.973+0000 7f2c5a7fc700 1 -- 172.21.15.83:0/3282274056 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c440035b0 con 0x7f2c5c0899e0 2023-11-28T11:08:54.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.969+0000 7f2c5b7fe700 1 --2- 172.21.15.83:0/3282274056 >> v2:172.21.15.90:3300/0 conn(0x7f2c5c071330 0x7f2c5c16cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.973+0000 7f2c5bfff700 1 -- 172.21.15.83:0/3282274056 >> v2:172.21.15.90:3300/0 conn(0x7f2c5c071330 msgr2=0x7f2c5c16cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.973+0000 7f2c5bfff700 1 --2- 172.21.15.83:0/3282274056 >> v2:172.21.15.90:3300/0 conn(0x7f2c5c071330 0x7f2c5c16cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.973+0000 7f2c5bfff700 1 -- 172.21.15.83:0/3282274056 >> v1:172.21.15.90:6789/0 conn(0x7f2c5c0899e0 legacy=0x7f2c5c0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.973+0000 7f2c5bfff700 1 -- 172.21.15.83:0/3282274056 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c071c00 con 0x7f2c5c0cc750 2023-11-28T11:08:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.973+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00fc30 con 0x7f8fe40041a0 2023-11-28T11:08:54.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.974+0000 7f2c5bfff700 1 --2- 172.21.15.83:0/3282274056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0cc750 0x7f2c5c0ccbd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c06f930 tx=0x7f2c540075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d0ea6b385bdc40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.974+0000 7f2c5a7fc700 1 -- 172.21.15.83:0/3282274056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c540094e0 con 0x7f2c5c0cc750 2023-11-28T11:08:54.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.974+0000 7f2c5a7fc700 1 -- 172.21.15.83:0/3282274056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c54009640 con 0x7f2c5c0cc750 2023-11-28T11:08:54.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.974+0000 7f2c5a7fc700 1 -- 172.21.15.83:0/3282274056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54003c60 con 0x7f2c5c0cc750 2023-11-28T11:08:54.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.974+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8fec00fd90 con 0x7f8fe40041a0 2023-11-28T11:08:54.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f8fe3fff700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8fe80682f0 0x7f8fe806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8fec086030 con 0x7f8fe40041a0 2023-11-28T11:08:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 -- 172.21.15.83:0/3282274056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0cc750 msgr2=0x7f2c5c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 --2- 172.21.15.83:0/3282274056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0cc750 0x7f2c5c0ccbd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c06f930 tx=0x7f2c540075f0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 -- 172.21.15.83:0/3282274056 shutdown_connections 2023-11-28T11:08:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 --2- 172.21.15.83:0/3282274056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0cc750 0x7f2c5c0ccbd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 --2- 172.21.15.83:0/3282274056 >> v2:172.21.15.90:3300/0 conn(0x7f2c5c071330 0x7f2c5c16cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 -- 172.21.15.83:0/3282274056 >> 172.21.15.83:0/3282274056 conn(0x7f2c5c0b1380 msgr2=0x7f2c5c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 -- 172.21.15.83:0/3282274056 shutdown_connections 2023-11-28T11:08:55.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc0052f0 con 0x7f8fe40041a0 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.975+0000 7f2c61e51700 1 -- 172.21.15.83:0/3282274056 wait complete. 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.977+0000 7f2c61e51700 1 Processor -- start 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.977+0000 7f2c61e51700 1 -- start start 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f2c61e51700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c5c071330 0x7f2c5c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f2c61e51700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 0x7f2c5c0a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f2c61e51700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c0cc750 0x7f2c5c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f2c61e51700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0a7540 con 0x7f2c5c0899e0 2023-11-28T11:08:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f2c61e51700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0a77e0 con 0x7f2c5c0cc750 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f2c61e51700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0a7ac0 con 0x7f2c5c071330 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 0x7f2c5c0a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 0x7f2c5c0a7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48964/0 (socket says 172.21.15.83:48964) 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 -- 172.21.15.83:0/2801650100 learned_addr learned my addr 172.21.15.83:0/2801650100 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5b7fe700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c5c071330 0x7f2c5c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 -- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c5c071330 msgr2=0x7f2c5c0c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c5c071330 0x7f2c5c0c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 -- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c0cc750 msgr2=0x7f2c5c0c7eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c0cc750 0x7f2c5c0c7eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 -- 172.21.15.83:0/2801650100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5400b050 con 0x7f2c5c0899e0 2023-11-28T11:08:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f8ffaffd700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8fe80682f0 0x7f8fe806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f8ffaffd700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8fe80682f0 0x7f8fe806a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc148ff0 tx=0x7f8ff0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f2c5affd700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 0x7f2c5c0a7eb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c001eb0 tx=0x7f2c4c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ff4010070 con 0x7f7004145430 2023-11-28T11:08:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f6feb7fe700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ff00683c0 0x7f6ff006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.978+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6ff4086120 con 0x7f7004145430 2023-11-28T11:08:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f7002d9d700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ff00683c0 0x7f6ff006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe40052f0 con 0x7f7004145430 2023-11-28T11:08:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.979+0000 7f7002d9d700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ff00683c0 0x7f6ff006a870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70040b17b0 tx=0x7f6ffc007270 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7fa56bdd2700 1 -- 172.21.15.83:0/3686695598 shutdown_connections 2023-11-28T11:08:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7fa56bdd2700 1 -- 172.21.15.83:0/3686695598 wait complete. 2023-11-28T11:08:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0054e0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c003680 con 0x7f2c5c0899e0 2023-11-28T11:08:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0039f0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ff4017080 con 0x7f7004145430 2023-11-28T11:08:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.985+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c16d8c0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.985+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c16d5e0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.987+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c4c019030 con 0x7f2c5c0899e0 2023-11-28T11:08:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7f2c43fff700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2c480683c0 0x7f2c4806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.985+0000 7fa56bdd2700 1 Processor -- start 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.986+0000 7fa56bdd2700 1 -- start start 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56bdd2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5640c21c0 0x7fa564148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56bdd2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640cce20 0x7fa56416d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56bdd2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 0x7fa5641622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56bdd2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c5700 con 0x7fa5640cce20 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56bdd2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa564149fc0 con 0x7fa564150c00 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56bdd2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56416d7c0 con 0x7fa5640c21c0 2023-11-28T11:08:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 0x7fa5641622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2c4c085290 con 0x7f2c5c0899e0 2023-11-28T11:08:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 0x7fa5641622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:35964/0 (socket says 172.21.15.83:35964) 2023-11-28T11:08:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 -- 172.21.15.83:0/2446798647 learned_addr learned my addr 172.21.15.83:0/2446798647 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56936d700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640cce20 0x7fa56416d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 -- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5640c21c0 msgr2=0x7fa564148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5640c21c0 0x7fa564148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 -- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640cce20 msgr2=0x7fa56416d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640cce20 0x7fa56416d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.988+0000 7fa56a36f700 1 -- 172.21.15.83:0/2446798647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c009b30 con 0x7fa564150c00 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.989+0000 7fa56a36f700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 0x7fa5641622e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00b370 tx=0x7fa55c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.989+0000 7f2c5b7fe700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2c480683c0 0x7f2c4806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.989+0000 7f2c5b7fe700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2c480683c0 0x7f2c4806a870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000b9b0 tx=0x7f2c50008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.984+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8fec0525a0 con 0x7f8fe40041a0 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.990+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0042c0 con 0x7fa564150c00 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.990+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c004420 con 0x7fa564150c00 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.990+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0047d0 con 0x7fa564150c00 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.990+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa564164680 con 0x7fa564150c00 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.990+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa564165290 con 0x7fa564150c00 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.992+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa55c012430 con 0x7fa564150c00 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.993+0000 7fa55a7fc700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5480683c0 0x7fa54806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.993+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa55c087430 con 0x7fa564150c00 2023-11-28T11:08:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.999+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c16dba0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.998+0000 7fa569b6e700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5480683c0 0x7fa54806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.998+0000 7fa569b6e700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5480683c0 0x7fa54806a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cf750 tx=0x7fa554006cd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:54.999+0000 7fa56936d700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640cce20 0x7fa56416d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.008+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c4c04b5e0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.011+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c0052f0 con 0x7fa564150c00 2023-11-28T11:08:55.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.016+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa55c057360 con 0x7fa564150c00 2023-11-28T11:08:55.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.046+0000 7fedcbbff700 1 Processor -- start 2023-11-28T11:08:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.046+0000 7fedcbbff700 1 -- start start 2023-11-28T11:08:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.047+0000 7fedcbbff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fedc406e440 0x7fedc415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedcbbff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedc4070ad0 0x7fedc4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedcbbff700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc415d2d0 con 0x7fedc4158790 2023-11-28T11:08:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedcbbff700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc4071490 con 0x7fedc4070ad0 2023-11-28T11:08:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedcbbff700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc406d7d0 con 0x7fedc406e440 2023-11-28T11:08:55.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedca19c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedc4070ad0 0x7fedc4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedca19c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedc4070ad0 0x7fedc4070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48978/0 (socket says 172.21.15.83:48978) 2023-11-28T11:08:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.048+0000 7fedca19c700 1 -- 172.21.15.83:0/3344261064 learned_addr learned my addr 172.21.15.83:0/3344261064 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.049+0000 7fedca19c700 1 -- 172.21.15.83:0/3344261064 >> v2:172.21.15.90:3300/0 conn(0x7fedc406e440 msgr2=0x7fedc415cd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.049+0000 7fedca19c700 1 --2- 172.21.15.83:0/3344261064 >> v2:172.21.15.90:3300/0 conn(0x7fedc406e440 0x7fedc415cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.049+0000 7fedca19c700 1 -- 172.21.15.83:0/3344261064 >> v1:172.21.15.83:6789/0 conn(0x7fedc4158790 legacy=0x7fedc406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.049+0000 7fedca19c700 1 -- 172.21.15.83:0/3344261064 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc406cea0 con 0x7fedc4070ad0 2023-11-28T11:08:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.049+0000 7fedca19c700 1 --2- 172.21.15.83:0/3344261064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc4070f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedc4071b40 tx=0x7fedbc00d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abaf7f386eefc9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.050+0000 7fedc8999700 1 -- 172.21.15.83:0/3344261064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc010070 con 0x7fedc4070ad0 2023-11-28T11:08:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.051+0000 7fedc8999700 1 -- 172.21.15.83:0/3344261064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc00dca0 con 0x7fedc4070ad0 2023-11-28T11:08:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.051+0000 7fedc8999700 1 -- 172.21.15.83:0/3344261064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc007d70 con 0x7fedc4070ad0 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.052+0000 7fedcbbff700 1 -- 172.21.15.83:0/3344261064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 msgr2=0x7fedc4070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.052+0000 7fedcbbff700 1 --2- 172.21.15.83:0/3344261064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc4070f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedc4071b40 tx=0x7fedbc00d650 comp rx=0 tx=0).stop 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.052+0000 7fedcbbff700 1 -- 172.21.15.83:0/3344261064 shutdown_connections 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.052+0000 7fedcbbff700 1 --2- 172.21.15.83:0/3344261064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc4070f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.052+0000 7fedcbbff700 1 --2- 172.21.15.83:0/3344261064 >> v2:172.21.15.90:3300/0 conn(0x7fedc406e440 0x7fedc415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.052+0000 7fedcbbff700 1 -- 172.21.15.83:0/3344261064 >> 172.21.15.83:0/3344261064 conn(0x7fedc40b1380 msgr2=0x7fedc40ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.053+0000 7fedcbbff700 1 -- 172.21.15.83:0/3344261064 shutdown_connections 2023-11-28T11:08:55.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.053+0000 7fedcbbff700 1 -- 172.21.15.83:0/3344261064 wait complete. 2023-11-28T11:08:55.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.054+0000 7fedcbbff700 1 Processor -- start 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.055+0000 7fedcbbff700 1 -- start start 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.056+0000 7fedcbbff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedc406e440 0x7fedc40c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.056+0000 7fedcbbff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc40c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.056+0000 7fedcbbff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedc4158790 0x7fedc40ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.056+0000 7fedcbbff700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40c8a30 con 0x7fedc4070ad0 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.056+0000 7fedcbbff700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40ca210 con 0x7fedc406e440 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.056+0000 7fedcbbff700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40c84b0 con 0x7fedc4158790 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc40c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc40c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48986/0 (socket says 172.21.15.83:48986) 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 -- 172.21.15.83:0/1123114583 learned_addr learned my addr 172.21.15.83:0/1123114583 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 -- 172.21.15.83:0/1123114583 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedc4158790 msgr2=0x7fedc40ca6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedc4158790 0x7fedc40ca6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 -- 172.21.15.83:0/1123114583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedc406e440 msgr2=0x7fedc40c4b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedc406e440 0x7fedc40c4b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.057+0000 7fedc919a700 1 -- 172.21.15.83:0/1123114583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc00d050 con 0x7fedc4070ad0 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.058+0000 7fedc919a700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc40c7f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b470 tx=0x7fedc000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.059+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011830 con 0x7fedc4070ad0 2023-11-28T11:08:55.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.059+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc00119b0 con 0x7fedc4070ad0 2023-11-28T11:08:55.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.059+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011d60 con 0x7fedc4070ad0 2023-11-28T11:08:55.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.059+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc406d460 con 0x7fedc4070ad0 2023-11-28T11:08:55.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.059+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc406d180 con 0x7fedc4070ad0 2023-11-28T11:08:55.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.064+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedc0021070 con 0x7fedc4070ad0 2023-11-28T11:08:55.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.064+0000 7fedba7fc700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb0068540 0x7fedb006a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.064+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fedc0085e10 con 0x7fedc4070ad0 2023-11-28T11:08:55.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.065+0000 7fedc999b700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb0068540 0x7fedb006a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.066+0000 7fedc999b700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb0068540 0x7fedb006a9f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fedb4006fd0 tx=0x7fedb4006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.079+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0052f0 con 0x7fedc4070ad0 2023-11-28T11:08:55.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.088+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedc0052310 con 0x7fedc4070ad0 2023-11-28T11:08:55.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.327+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fba580055a0 con 0x7fba7c0c2750 2023-11-28T11:08:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.327+0000 7fba78ff9700 1 -- 172.21.15.83:0/2303686650 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba6c017090 con 0x7fba7c0c2750 2023-11-28T11:08:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-28T11:08:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.331+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fba60068310 msgr2=0x7fba6006a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.331+0000 7fba82518700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fba60068310 0x7fba6006a7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba74004c50 tx=0x7fba74006040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.331+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 msgr2=0x7fba7c1f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.331+0000 7fba82518700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 0x7fba7c1f90f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba6c016000 tx=0x7fba6c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 shutdown_connections 2023-11-28T11:08:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c084420 0x7fba7c1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fba60068310 0x7fba6006a7c0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba74004c50 tx=0x7fba74006040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c0c2750 0x7fba7c1f90f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 --2- 172.21.15.83:0/2303686650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fba7c0c2b20 0x7fba7c0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 >> 172.21.15.83:0/2303686650 conn(0x7fba7c0b1380 msgr2=0x7fba7c16ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 shutdown_connections 2023-11-28T11:08:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.332+0000 7fba82518700 1 -- 172.21.15.83:0/2303686650 wait complete. 2023-11-28T11:08:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.344+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faa100b3210 con 0x7faa10158790 2023-11-28T11:08:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.344+0000 7faa0cff9700 1 -- 172.21.15.83:0/1828739153 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa100b3210 con 0x7faa10158790 2023-11-28T11:08:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-28T11:08:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.348+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa9fc0683c0 msgr2=0x7fa9fc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.348+0000 7faa16372700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa9fc0683c0 0x7fa9fc06a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa000050b0 tx=0x7faa00004f20 comp rx=0 tx=0).stop 2023-11-28T11:08:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.348+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 msgr2=0x7faa100b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.348+0000 7faa16372700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 0x7faa100b34f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa100715d0 tx=0x7faa08004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.349+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 shutdown_connections 2023-11-28T11:08:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.349+0000 7faa16372700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa9fc0683c0 0x7fa9fc06a870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.349+0000 7faa16372700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa10158790 0x7faa100b34f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.349+0000 7faa16372700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa1006e440 0x7faa100b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.349+0000 7faa16372700 1 --2- 172.21.15.83:0/1828739153 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faa10070ad0 0x7faa100af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.349+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 >> 172.21.15.83:0/1828739153 conn(0x7faa100b1380 msgr2=0x7faa1006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.1 2023-11-28T11:08:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.350+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f62b00055a0 con 0x7f62d00cf290 2023-11-28T11:08:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.350+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 shutdown_connections 2023-11-28T11:08:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.350+0000 7faa16372700 1 -- 172.21.15.83:0/1828739153 wait complete. 2023-11-28T11:08:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.352+0000 7f62cd7fa700 1 -- 172.21.15.83:0/112695953 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62c803c020 con 0x7f62d00cf290 2023-11-28T11:08:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-28T11:08:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.354+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62bc0682f0 msgr2=0x7f62bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.354+0000 7f62d6f15700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62bc0682f0 0x7f62bc06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62d00c7870 tx=0x7f62c0009190 comp rx=0 tx=0).stop 2023-11-28T11:08:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.354+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 msgr2=0x7f62d00c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.354+0000 7f62d6f15700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 0x7f62d00c4880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62c8026040 tx=0x7f62c8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.359+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 shutdown_connections 2023-11-28T11:08:55.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.359+0000 7f62d6f15700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62bc0682f0 0x7f62bc06a7a0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62d00c7870 tx=0x7f62c0009190 comp rx=0 tx=0).stop 2023-11-28T11:08:55.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.359+0000 7f62d6f15700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62d015cd90 0x7f62d00c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.359+0000 7f62d6f15700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d00cf290 0x7f62d00c4880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.359+0000 7f62d6f15700 1 --2- 172.21.15.83:0/112695953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d0158520 0x7f62d00c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.359+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 >> 172.21.15.83:0/112695953 conn(0x7f62d00b1380 msgr2=0x7f62d0159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.363+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 shutdown_connections 2023-11-28T11:08:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.364+0000 7f62d6f15700 1 -- 172.21.15.83:0/112695953 wait complete. 2023-11-28T11:08:55.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.366+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6fe40055a0 con 0x7f7004145430 2023-11-28T11:08:55.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.367+0000 7f6feb7fe700 1 -- 172.21.15.83:0/2689899935 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ff4029070 con 0x7f7004145430 2023-11-28T11:08:55.372 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:55.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.4 2023-11-28T11:08:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.371+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8fdc005160 con 0x7f8fe40041a0 2023-11-28T11:08:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.371+0000 7f8fe3fff700 1 -- 172.21.15.83:0/3716561908 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8fec055bc0 con 0x7f8fe40041a0 2023-11-28T11:08:55.375 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-28T11:08:55.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.373+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8fe80682f0 msgr2=0x7f8fe806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.373+0000 7f9001c78700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8fe80682f0 0x7f8fe806a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc148ff0 tx=0x7f8ff0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.373+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 msgr2=0x7f8ffc0c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.373+0000 7f9001c78700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8ffc0c26b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0017a0 tx=0x7f8fec0075c0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 shutdown_connections 2023-11-28T11:08:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ffc0c50e0 0x7f8ffc0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8fe80682f0 0x7f8fe806a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fe40041a0 0x7f8ffc0c26b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 --2- 172.21.15.83:0/3716561908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc0c2bf0 0x7f8ffc0c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 >> 172.21.15.83:0/3716561908 conn(0x7f8ffc0b1380 msgr2=0x7f8ffc161150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 shutdown_connections 2023-11-28T11:08:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.375+0000 7f9001c78700 1 -- 172.21.15.83:0/3716561908 wait complete. 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.376+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ff00683c0 msgr2=0x7f6ff006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.376+0000 7f7009050700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ff00683c0 0x7f6ff006a870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70040b17b0 tx=0x7f6ffc007270 comp rx=0 tx=0).stop 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.376+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004145430 msgr2=0x7f70040b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.376+0000 7f7009050700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004145430 0x7f70040b4470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c240 tx=0x7f6ff4001930 comp rx=0 tx=0).stop 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.377+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 shutdown_connections 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.377+0000 7f7009050700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7004145430 0x7f70040b4470 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.377+0000 7f7009050700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7004145060 0x7f70040b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.377+0000 7f7009050700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ff00683c0 0x7f6ff006a870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.377+0000 7f7009050700 1 --2- 172.21.15.83:0/2689899935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70040b3120 0x7f70040b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.377+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 >> 172.21.15.83:0/2689899935 conn(0x7f70040b1380 msgr2=0x7f7004158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.0 2023-11-28T11:08:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.378+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa54c0055a0 con 0x7fa564150c00 2023-11-28T11:08:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.378+0000 7fa55a7fc700 1 -- 172.21.15.83:0/2446798647 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa55c017070 con 0x7fa564150c00 2023-11-28T11:08:55.383 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-28T11:08:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5480683c0 msgr2=0x7fa54806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5480683c0 0x7fa54806a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cf750 tx=0x7fa554006cd0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 msgr2=0x7fa5641622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 0x7fa5641622e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00b370 tx=0x7fa55c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 shutdown_connections 2023-11-28T11:08:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 shutdown_connections 2023-11-28T11:08:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa564150c00 0x7fa5641622e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5640c21c0 0x7fa564148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa5480683c0 0x7fa54806a870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 --2- 172.21.15.83:0/2446798647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5640cce20 0x7fa56416d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7f7009050700 1 -- 172.21.15.83:0/2689899935 wait complete. 2023-11-28T11:08:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.381+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 >> 172.21.15.83:0/2446798647 conn(0x7fa5640ceb80 msgr2=0x7fa5640c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.382+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 shutdown_connections 2023-11-28T11:08:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.382+0000 7fa56bdd2700 1 -- 172.21.15.83:0/2446798647 wait complete. 2023-11-28T11:08:55.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.393+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2c5c0a7ce0 con 0x7f2c5c0899e0 2023-11-28T11:08:55.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.393+0000 7f2c43fff700 1 -- 172.21.15.83:0/2801650100 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c4c017020 con 0x7f2c5c0899e0 2023-11-28T11:08:55.395 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-28T11:08:55.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.7 2023-11-28T11:08:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2c480683c0 msgr2=0x7f2c4806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2c480683c0 0x7f2c4806a870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000b9b0 tx=0x7f2c50008040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 msgr2=0x7f2c5c0a7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 0x7f2c5c0a7eb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c001eb0 tx=0x7f2c4c00c8b0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 shutdown_connections 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c0cc750 0x7f2c5c0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c5c071330 0x7f2c5c0c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2c480683c0 0x7f2c4806a870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 --2- 172.21.15.83:0/2801650100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c0899e0 0x7f2c5c0a7eb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 >> 172.21.15.83:0/2801650100 conn(0x7f2c5c0b1380 msgr2=0x7f2c5c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 shutdown_connections 2023-11-28T11:08:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.396+0000 7f2c61e51700 1 -- 172.21.15.83:0/2801650100 wait complete. 2023-11-28T11:08:55.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2023-11-28T11:08:55.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.6 2023-11-28T11:08:55.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.3 2023-11-28T11:08:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.436+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fedac0055a0 con 0x7fedc4070ad0 2023-11-28T11:08:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.437+0000 7fedba7fc700 1 -- 172.21.15.83:0/1123114583 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fedc0015020 con 0x7fedc4070ad0 2023-11-28T11:08:55.438 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb0068540 msgr2=0x7fedb006a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb0068540 0x7fedb006a9f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fedb4006fd0 tx=0x7fedb4006bc0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 msgr2=0x7fedc40c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc40c7f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b470 tx=0x7fedc000b830 comp rx=0 tx=0).stop 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 shutdown_connections 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedc4158790 0x7fedc40ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedc406e440 0x7fedc40c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb0068540 0x7fedb006a9f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 --2- 172.21.15.83:0/1123114583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc4070ad0 0x7fedc40c7f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 >> 172.21.15.83:0/1123114583 conn(0x7fedc40b1380 msgr2=0x7fedc415b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 shutdown_connections 2023-11-28T11:08:55.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:55.439+0000 7fedcbbff700 1 -- 172.21.15.83:0/1123114583 wait complete. 2023-11-28T11:08:55.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2023-11-28T11:08:56.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:08:56.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:08:56.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:08:56.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:08:56.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:08:56.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:08:56.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:08:56.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:08:56.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.510+0000 7f59eaab1700 1 Processor -- start 2023-11-28T11:08:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.510+0000 7f59eaab1700 1 -- start start 2023-11-28T11:08:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.512+0000 7f59eaab1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59e406e440 0x7f59e415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.512+0000 7f59eaab1700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f59e4158700 0x7f59e4070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.512+0000 7f59eaab1700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e415d2d0 con 0x7f59e40cf290 2023-11-28T11:08:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.512+0000 7f59eaab1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e406d7d0 con 0x7f59e406e440 2023-11-28T11:08:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.512+0000 7f59eaab1700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e406cea0 con 0x7f59e4158700 2023-11-28T11:08:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.512+0000 7f59e904e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f59e4158700 0x7f59e4070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e884d700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f59e40cf290 0x7f59e406dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45198/0 (socket says 172.21.15.83:45198) 2023-11-28T11:08:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e884d700 1 -- 172.21.15.83:0/784205769 learned_addr learned my addr 172.21.15.83:0/784205769 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e3fff700 1 --2- 172.21.15.83:0/784205769 >> v2:172.21.15.83:3300/0 conn(0x7f59e406e440 0x7f59e415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e904e700 1 -- 172.21.15.83:0/784205769 >> v2:172.21.15.83:3300/0 conn(0x7f59e406e440 msgr2=0x7f59e415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e904e700 1 --2- 172.21.15.83:0/784205769 >> v2:172.21.15.83:3300/0 conn(0x7f59e406e440 0x7f59e415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e904e700 1 -- 172.21.15.83:0/784205769 >> v1:172.21.15.90:6789/0 conn(0x7f59e40cf290 legacy=0x7f59e406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.513+0000 7f59e904e700 1 -- 172.21.15.83:0/784205769 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e406d180 con 0x7f59e4158700 2023-11-28T11:08:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.514+0000 7f59e904e700 1 --2- 172.21.15.83:0/784205769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e4158700 0x7f59e4070ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e40ae5a0 tx=0x7f59d4009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544b5e5aef14be5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.514+0000 7f59e37fe700 1 -- 172.21.15.83:0/784205769 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011830 con 0x7f59e4158700 2023-11-28T11:08:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.515+0000 7f59e37fe700 1 -- 172.21.15.83:0/784205769 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4011990 con 0x7f59e4158700 2023-11-28T11:08:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.515+0000 7f59e37fe700 1 -- 172.21.15.83:0/784205769 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011d20 con 0x7f59e4158700 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.515+0000 7f59eaab1700 1 -- 172.21.15.83:0/784205769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e4158700 msgr2=0x7f59e4070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.515+0000 7f59eaab1700 1 --2- 172.21.15.83:0/784205769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e4158700 0x7f59e4070ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e40ae5a0 tx=0x7f59d4009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.516+0000 7f59eaab1700 1 -- 172.21.15.83:0/784205769 shutdown_connections 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.516+0000 7f59eaab1700 1 --2- 172.21.15.83:0/784205769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e4158700 0x7f59e4070ad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.516+0000 7f59eaab1700 1 --2- 172.21.15.83:0/784205769 >> v2:172.21.15.83:3300/0 conn(0x7f59e406e440 0x7f59e415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.516+0000 7f59eaab1700 1 -- 172.21.15.83:0/784205769 >> 172.21.15.83:0/784205769 conn(0x7f59e40b1380 msgr2=0x7f59e40ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.516+0000 7f59eaab1700 1 -- 172.21.15.83:0/784205769 shutdown_connections 2023-11-28T11:08:56.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.516+0000 7f59eaab1700 1 -- 172.21.15.83:0/784205769 wait complete. 2023-11-28T11:08:56.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.518+0000 7f59eaab1700 1 Processor -- start 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.518+0000 7f59eaab1700 1 -- start start 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59eaab1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e406e440 0x7f59e40c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59eaab1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59e40cf290 0x7f59e40b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59eaab1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 0x7f59e41f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59eaab1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40b4390 con 0x7f59e4158700 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59eaab1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40b3a20 con 0x7f59e406e440 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59eaab1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e414c760 con 0x7f59e40cf290 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.519+0000 7f59e904e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 0x7f59e41f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 0x7f59e41f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49000/0 (socket says 172.21.15.83:49000) 2023-11-28T11:08:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 -- 172.21.15.83:0/2569239941 learned_addr learned my addr 172.21.15.83:0/2569239941 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e3fff700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59e40cf290 0x7f59e40b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 -- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59e40cf290 msgr2=0x7f59e40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59e40cf290 0x7f59e40b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 -- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e406e440 msgr2=0x7f59e40c8450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e406e440 0x7f59e40c8450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 -- 172.21.15.83:0/2569239941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40094a0 con 0x7f59e4158700 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.520+0000 7f59e904e700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 0x7f59e41f8630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f59d40090b0 tx=0x7f59d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.521+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40036a0 con 0x7f59e4158700 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.521+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d401e070 con 0x7f59e4158700 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.521+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4003a20 con 0x7f59e4158700 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.521+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e4158b80 con 0x7f59e4158700 2023-11-28T11:08:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.521+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e406d460 con 0x7f59e4158700 2023-11-28T11:08:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.524+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f59d400f050 con 0x7f59e4158700 2023-11-28T11:08:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.525+0000 7f59e17fa700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59d0068340 0x7f59d006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.525+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f59d4086e30 con 0x7f59e4158700 2023-11-28T11:08:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.526+0000 7f59e884d700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59d0068340 0x7f59d006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.526+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c40052f0 con 0x7f59e4158700 2023-11-28T11:08:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.536+0000 7f59e884d700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59d0068340 0x7f59d006a7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59e40b01b0 tx=0x7f59dc0015f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.536+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f59d4020080 con 0x7f59e4158700 2023-11-28T11:08:56.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.574+0000 7f3015355700 1 Processor -- start 2023-11-28T11:08:56.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.574+0000 7f3015355700 1 -- start start 2023-11-28T11:08:56.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f3015355700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3010158790 0x7f301006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f3015355700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3010070ad0 0x7f3010070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f3015355700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301006d7d0 con 0x7f301015cd90 2023-11-28T11:08:56.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f3015355700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3010071490 con 0x7f3010158790 2023-11-28T11:08:56.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f3015355700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301006cea0 con 0x7f3010070ad0 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f300e7fc700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f301015cd90 0x7f301006dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45212/0 (socket says 172.21.15.83:45212) 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.576+0000 7f300e7fc700 1 -- 172.21.15.83:0/1847117480 learned_addr learned my addr 172.21.15.83:0/1847117480 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300f7fe700 1 --2- 172.21.15.83:0/1847117480 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3010070ad0 0x7f3010070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300dffb700 1 -- 172.21.15.83:0/1847117480 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056863444 0 0) 0x7f301006d7d0 con 0x7f301015cd90 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300dffb700 1 -- 172.21.15.83:0/1847117480 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff80035b0 con 0x7f301015cd90 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300effd700 1 --2- 172.21.15.83:0/1847117480 >> v2:172.21.15.83:3300/0 conn(0x7f3010158790 0x7f301006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300f7fe700 1 -- 172.21.15.83:0/1847117480 >> v2:172.21.15.83:3300/0 conn(0x7f3010158790 msgr2=0x7f301006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300f7fe700 1 --2- 172.21.15.83:0/1847117480 >> v2:172.21.15.83:3300/0 conn(0x7f3010158790 0x7f301006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300f7fe700 1 -- 172.21.15.83:0/1847117480 >> v1:172.21.15.90:6789/0 conn(0x7f301015cd90 legacy=0x7f301006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300dffb700 1 -- 172.21.15.83:0/1847117480 <== mon.1 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2500371819 0 0) 0x7f2ff80035b0 con 0x7f301015cd90 2023-11-28T11:08:56.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300f7fe700 1 -- 172.21.15.83:0/1847117480 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301006d180 con 0x7f3010070ad0 2023-11-28T11:08:56.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300f7fe700 1 --2- 172.21.15.83:0/1847117480 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3010070ad0 0x7f3010070f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f300800aea0 tx=0x7f300800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd1a9d41485a2c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.577+0000 7f300dffb700 1 -- 172.21.15.83:0/1847117480 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000004ef0 con 0x7f3010070ad0 2023-11-28T11:08:56.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.578+0000 7f300dffb700 1 -- 172.21.15.83:0/1847117480 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300800a0d0 con 0x7f3010070ad0 2023-11-28T11:08:56.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.578+0000 7f300dffb700 1 -- 172.21.15.83:0/1847117480 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008003a20 con 0x7f3010070ad0 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 -- 172.21.15.83:0/1847117480 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3010070ad0 msgr2=0x7f3010070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 --2- 172.21.15.83:0/1847117480 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3010070ad0 0x7f3010070f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f300800aea0 tx=0x7f300800c330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 -- 172.21.15.83:0/1847117480 shutdown_connections 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 --2- 172.21.15.83:0/1847117480 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3010070ad0 0x7f3010070f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 --2- 172.21.15.83:0/1847117480 >> v2:172.21.15.83:3300/0 conn(0x7f3010158790 0x7f301006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 -- 172.21.15.83:0/1847117480 >> 172.21.15.83:0/1847117480 conn(0x7f30100b1380 msgr2=0x7f30100ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 -- 172.21.15.83:0/1847117480 shutdown_connections 2023-11-28T11:08:56.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.579+0000 7f3015355700 1 -- 172.21.15.83:0/1847117480 wait complete. 2023-11-28T11:08:56.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.581+0000 7f11c1a97700 1 Processor -- start 2023-11-28T11:08:56.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.581+0000 7f11c1a97700 1 -- start start 2023-11-28T11:08:56.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.581+0000 7f3015355700 1 Processor -- start 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f3015355700 1 -- start start 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f3015355700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3010070ad0 0x7f30101ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f3015355700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 0x7f30100c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f3015355700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f301015cd90 0x7f30100c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f3015355700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101efc80 con 0x7f3010158790 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f3015355700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101eedd0 con 0x7f3010070ad0 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f3015355700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30101ef070 con 0x7f301015cd90 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f300e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 0x7f30100c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f300e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 0x7f30100c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49018/0 (socket says 172.21.15.83:49018) 2023-11-28T11:08:56.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f300e7fc700 1 -- 172.21.15.83:0/1715247399 learned_addr learned my addr 172.21.15.83:0/1715247399 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f11c1a97700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc158790 0x7f11bc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f11c1a97700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f11bc15cd90 0x7f11bc06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f11c1a97700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11bc070ad0 0x7f11bc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f11c1a97700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc071490 con 0x7f11bc070ad0 2023-11-28T11:08:56.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f11c1a97700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc06d7d0 con 0x7f11bc15cd90 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.582+0000 7f11c1a97700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc06e1a0 con 0x7f11bc158790 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11bc070ad0 0x7f11bc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11bc070ad0 0x7f11bc070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49016/0 (socket says 172.21.15.83:49016) 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 -- 172.21.15.83:0/1393281322 learned_addr learned my addr 172.21.15.83:0/1393281322 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 -- 172.21.15.83:0/1393281322 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc158790 msgr2=0x7f11bc06e440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 --2- 172.21.15.83:0/1393281322 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc158790 0x7f11bc06e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 -- 172.21.15.83:0/1393281322 >> v2:172.21.15.90:3300/0 conn(0x7f11bc15cd90 msgr2=0x7f11bc06dc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 --2- 172.21.15.83:0/1393281322 >> v2:172.21.15.90:3300/0 conn(0x7f11bc15cd90 0x7f11bc06dc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 -- 172.21.15.83:0/1393281322 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc06cea0 con 0x7f11bc070ad0 2023-11-28T11:08:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.583+0000 7f11bbfff700 1 --2- 172.21.15.83:0/1393281322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc070ad0 0x7f11bc070f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0cf6e0 tx=0x7f11a4008bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61a49ed17717415 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.584+0000 7f300e7fc700 1 -- 172.21.15.83:0/1715247399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f301015cd90 msgr2=0x7f30100c87a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:56.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.584+0000 7f300e7fc700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f301015cd90 0x7f30100c87a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.584+0000 7f300e7fc700 1 -- 172.21.15.83:0/1715247399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3010070ad0 msgr2=0x7f30101ee4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:56.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.584+0000 7f300e7fc700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3010070ad0 0x7f30101ee4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.584+0000 7f300e7fc700 1 -- 172.21.15.83:0/1715247399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300800ab80 con 0x7f3010158790 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.584+0000 7f300e7fc700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 0x7f30100c8260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f300000bc60 tx=0x7f3000009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.585+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300000cd80 con 0x7f3010158790 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.585+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000015470 con 0x7f3010158790 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.585+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000015820 con 0x7f3010158790 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.586+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301015d1e0 con 0x7f3010158790 2023-11-28T11:08:56.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.586+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301006d460 con 0x7f3010158790 2023-11-28T11:08:56.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.588+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3010158b60 con 0x7f3010158790 2023-11-28T11:08:56.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.588+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3000015980 con 0x7f3010158790 2023-11-28T11:08:56.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.589+0000 7f2ff77fe700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2ffc068340 0x7f2ffc06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.589+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3000086330 con 0x7f3010158790 2023-11-28T11:08:56.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.589+0000 7f300effd700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2ffc068340 0x7f2ffc06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.590+0000 7f300effd700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2ffc068340 0x7f2ffc06a7f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f300400b240 tx=0x7f300400b420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.597+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3000052990 con 0x7f3010158790 2023-11-28T11:08:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.600+0000 7f11ba7fc700 1 -- 172.21.15.83:0/1393281322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a400b820 con 0x7f11bc070ad0 2023-11-28T11:08:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.600+0000 7f11ba7fc700 1 -- 172.21.15.83:0/1393281322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a400b980 con 0x7f11bc070ad0 2023-11-28T11:08:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.600+0000 7f11ba7fc700 1 -- 172.21.15.83:0/1393281322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a40126f0 con 0x7f11bc070ad0 2023-11-28T11:08:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.601+0000 7f11c1a97700 1 -- 172.21.15.83:0/1393281322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc070ad0 msgr2=0x7f11bc070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.601+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1393281322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc070ad0 0x7f11bc070f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0cf6e0 tx=0x7f11a4008bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 -- 172.21.15.83:0/1393281322 shutdown_connections 2023-11-28T11:08:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1393281322 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc158790 0x7f11bc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1393281322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc070ad0 0x7f11bc070f50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1393281322 >> v2:172.21.15.90:3300/0 conn(0x7f11bc15cd90 0x7f11bc06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 -- 172.21.15.83:0/1393281322 >> 172.21.15.83:0/1393281322 conn(0x7f11bc0b1380 msgr2=0x7f11bc0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 -- 172.21.15.83:0/1393281322 shutdown_connections 2023-11-28T11:08:56.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.602+0000 7f11c1a97700 1 -- 172.21.15.83:0/1393281322 wait complete. 2023-11-28T11:08:56.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.604+0000 7f11c1a97700 1 Processor -- start 2023-11-28T11:08:56.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.605+0000 7f11c1a97700 1 -- start start 2023-11-28T11:08:56.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11c1a97700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11bc070ad0 0x7f11bc150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11c1a97700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 0x7f11bc0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11c1a97700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc15cd90 0x7f11bc150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11c1a97700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0b3900 con 0x7f11bc158790 2023-11-28T11:08:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11c1a97700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0b3fe0 con 0x7f11bc070ad0 2023-11-28T11:08:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11c1a97700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0b42c0 con 0x7f11bc15cd90 2023-11-28T11:08:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11baffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 0x7f11bc0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11baffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 0x7f11bc0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49028/0 (socket says 172.21.15.83:49028) 2023-11-28T11:08:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11baffd700 1 -- 172.21.15.83:0/1946636579 learned_addr learned my addr 172.21.15.83:0/1946636579 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.606+0000 7f11bb7fe700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11bc070ad0 0x7f11bc150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11bb7fe700 1 -- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc15cd90 msgr2=0x7f11bc150960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11bb7fe700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc15cd90 0x7f11bc150960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11bb7fe700 1 -- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 msgr2=0x7f11bc0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11bb7fe700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 0x7f11bc0b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11bb7fe700 1 -- 172.21.15.83:0/1946636579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40085d0 con 0x7f11bc070ad0 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11baffd700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 0x7f11bc0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.607+0000 7f11bb7fe700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11bc070ad0 0x7f11bc150110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11b400be20 tx=0x7f11b4009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.608+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b400cd10 con 0x7f11bc070ad0 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.608+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b400ce70 con 0x7f11bc070ad0 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.608+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4014660 con 0x7f11bc070ad0 2023-11-28T11:08:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.608+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc06d460 con 0x7f11bc070ad0 2023-11-28T11:08:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.608+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc06d180 con 0x7f11bc070ad0 2023-11-28T11:08:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.610+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f11b401b030 con 0x7f11bc070ad0 2023-11-28T11:08:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.611+0000 7f11b3fff700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f119c0683c0 0x7f119c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.611+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f11b4085e00 con 0x7f11bc070ad0 2023-11-28T11:08:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.611+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a00052f0 con 0x7f11bc070ad0 2023-11-28T11:08:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.617+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f11b40522a0 con 0x7f11bc070ad0 2023-11-28T11:08:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.618+0000 7f11baffd700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f119c0683c0 0x7f119c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.619+0000 7f11baffd700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f119c0683c0 0x7f119c06a870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0c5000 tx=0x7f11ac009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.645+0000 7f4de759e700 1 Processor -- start 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.645+0000 7f4de759e700 1 -- start start 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.646+0000 7f4de759e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80ccb70 0x7f4de816ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.646+0000 7f4de759e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4de80710b0 0x7f4de806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.646+0000 7f4de759e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de80ccf40 con 0x7f4de806b7b0 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.646+0000 7f4de759e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de816d490 con 0x7f4de80710b0 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.646+0000 7f4de759e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de816d770 con 0x7f4de80ccb70 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.647+0000 7f4de5d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4de806b7b0 0x7f4de80709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46898/0 (socket says 172.21.15.83:46898) 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.647+0000 7f4de5d9b700 1 -- 172.21.15.83:0/2355908894 learned_addr learned my addr 172.21.15.83:0/2355908894 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.647+0000 7f4de6d9d700 1 --2- 172.21.15.83:0/2355908894 >> v2:172.21.15.90:3300/0 conn(0x7f4de80710b0 0x7f4de806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.647+0000 7f4de659c700 1 --2- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80ccb70 0x7f4de816ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.648+0000 7f4de6d9d700 1 -- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80ccb70 msgr2=0x7f4de816ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.648+0000 7f4de6d9d700 1 --2- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80ccb70 0x7f4de816ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.648+0000 7f4de6d9d700 1 -- 172.21.15.83:0/2355908894 >> v1:172.21.15.83:6789/0 conn(0x7f4de806b7b0 legacy=0x7f4de80709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.648+0000 7f4de6d9d700 1 -- 172.21.15.83:0/2355908894 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de816da50 con 0x7f4de80710b0 2023-11-28T11:08:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.649+0000 7f45baef6700 1 Processor -- start 2023-11-28T11:08:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.649+0000 7f45baef6700 1 -- start start 2023-11-28T11:08:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45baef6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45b406acd0 0x7f45b40c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45baef6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b406b0a0 0x7f45b4070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45baef6700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b40c97c0 con 0x7f45b41651e0 2023-11-28T11:08:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45baef6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b4165750 con 0x7f45b406acd0 2023-11-28T11:08:56.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45baef6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c9aa0 con 0x7f45b406b0a0 2023-11-28T11:08:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45b8c92700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45b406acd0 0x7f45b40c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45b8c92700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45b406acd0 0x7f45b40c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49042/0 (socket says 172.21.15.83:49042) 2023-11-28T11:08:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45b8c92700 1 -- 172.21.15.83:0/3916263157 learned_addr learned my addr 172.21.15.83:0/3916263157 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.650+0000 7f45b9493700 1 --1- 172.21.15.83:0/3916263157 >> v1:172.21.15.90:6789/0 conn(0x7f45b41651e0 0x7f45b40ccba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45246/0 (socket says 172.21.15.83:45246) 2023-11-28T11:08:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b37fe700 1 -- 172.21.15.83:0/3916263157 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993339355 0 0) 0x7f45b40c97c0 con 0x7f45b41651e0 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b37fe700 1 -- 172.21.15.83:0/3916263157 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f459c0035b0 con 0x7f45b41651e0 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b8c92700 1 -- 172.21.15.83:0/3916263157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b406b0a0 msgr2=0x7f45b4070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b8c92700 1 --2- 172.21.15.83:0/3916263157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b406b0a0 0x7f45b4070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b8c92700 1 -- 172.21.15.83:0/3916263157 >> v1:172.21.15.90:6789/0 conn(0x7f45b41651e0 legacy=0x7f45b40ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b8c92700 1 -- 172.21.15.83:0/3916263157 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b40cb480 con 0x7f45b406acd0 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b8c92700 1 --2- 172.21.15.83:0/3916263157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b40c1d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098f0 tx=0x7f45a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38450ebf1a82bd3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b37fe700 1 -- 172.21.15.83:0/3916263157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4011830 con 0x7f45b406acd0 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b37fe700 1 -- 172.21.15.83:0/3916263157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4011990 con 0x7f45b406acd0 2023-11-28T11:08:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.651+0000 7f45b37fe700 1 -- 172.21.15.83:0/3916263157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4011d00 con 0x7f45b406acd0 2023-11-28T11:08:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.652+0000 7f45baef6700 1 -- 172.21.15.83:0/3916263157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 msgr2=0x7f45b40c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.652+0000 7f45baef6700 1 --2- 172.21.15.83:0/3916263157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b40c1d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098f0 tx=0x7f45a4009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.652+0000 7f45baef6700 1 -- 172.21.15.83:0/3916263157 shutdown_connections 2023-11-28T11:08:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.652+0000 7f45baef6700 1 --2- 172.21.15.83:0/3916263157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b40c1d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.652+0000 7f45baef6700 1 --2- 172.21.15.83:0/3916263157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b406b0a0 0x7f45b4070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.652+0000 7f45baef6700 1 -- 172.21.15.83:0/3916263157 >> 172.21.15.83:0/3916263157 conn(0x7f45b40cba50 msgr2=0x7f45b40cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.655+0000 7f2fac6f7700 1 Processor -- start 2023-11-28T11:08:56.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.655+0000 7f4de6d9d700 1 --2- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de80710b0 0x7f4de806cfc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4de000bfa0 tx=0x7f4de000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efdeaeabc9c449e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.655+0000 7f4de559a700 1 -- 172.21.15.83:0/2355908894 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0010070 con 0x7f4de80710b0 2023-11-28T11:08:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.655+0000 7f4de559a700 1 -- 172.21.15.83:0/2355908894 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000b2e0 con 0x7f4de80710b0 2023-11-28T11:08:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.655+0000 7f4de559a700 1 -- 172.21.15.83:0/2355908894 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0003c60 con 0x7f4de80710b0 2023-11-28T11:08:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.656+0000 7f4de759e700 1 -- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de80710b0 msgr2=0x7f4de806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.656+0000 7f4de759e700 1 --2- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de80710b0 0x7f4de806cfc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4de000bfa0 tx=0x7f4de000d330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f4de759e700 1 -- 172.21.15.83:0/2355908894 shutdown_connections 2023-11-28T11:08:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f4de759e700 1 --2- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de80710b0 0x7f4de806cfc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f4de759e700 1 --2- 172.21.15.83:0/2355908894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80ccb70 0x7f4de816ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f4de759e700 1 -- 172.21.15.83:0/2355908894 >> 172.21.15.83:0/2355908894 conn(0x7f4de80cea00 msgr2=0x7f4de80cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f4de759e700 1 -- 172.21.15.83:0/2355908894 shutdown_connections 2023-11-28T11:08:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f4de759e700 1 -- 172.21.15.83:0/2355908894 wait complete. 2023-11-28T11:08:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.655+0000 7f2fac6f7700 1 -- start start 2023-11-28T11:08:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2fac6f7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 0x7f2fa4160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2fac6f7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fa40d0d10 0x7f2fa40d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2fac6f7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa40cd5f0 con 0x7f2fa406b3a0 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2fac6f7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa41610f0 con 0x7f2fa40d0d10 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2fac6f7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa40cf150 con 0x7f2fa40b4980 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2faa493700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 0x7f2fa4160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2faa493700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 0x7f2fa4160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:33248/0 (socket says 172.21.15.83:33248) 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.657+0000 7f2faa493700 1 -- 172.21.15.83:0/3393739356 learned_addr learned my addr 172.21.15.83:0/3393739356 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.658+0000 7f2faa493700 1 -- 172.21.15.83:0/3393739356 >> v2:172.21.15.83:3300/0 conn(0x7f2fa40d0d10 msgr2=0x7f2fa40d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.658+0000 7f2faa493700 1 --2- 172.21.15.83:0/3393739356 >> v2:172.21.15.83:3300/0 conn(0x7f2fa40d0d10 0x7f2fa40d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.658+0000 7f2faa493700 1 -- 172.21.15.83:0/3393739356 >> v1:172.21.15.83:6789/0 conn(0x7f2fa406b3a0 legacy=0x7f2fa40b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.658+0000 7f2faa493700 1 -- 172.21.15.83:0/3393739356 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa40cf3f0 con 0x7f2fa40b4980 2023-11-28T11:08:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.659+0000 7f2faa493700 1 --2- 172.21.15.83:0/3393739356 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 0x7f2fa4160bb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009e50 tx=0x7f2f9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48f10e78704b387 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.659+0000 7f2fa9491700 1 -- 172.21.15.83:0/3393739356 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00e070 con 0x7f2fa40b4980 2023-11-28T11:08:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.659+0000 7f2fa9491700 1 -- 172.21.15.83:0/3393739356 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c003a20 con 0x7f2fa40b4980 2023-11-28T11:08:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.659+0000 7f2fa9491700 1 -- 172.21.15.83:0/3393739356 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c003dd0 con 0x7f2fa40b4980 2023-11-28T11:08:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.659+0000 7f4de759e700 1 Processor -- start 2023-11-28T11:08:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.659+0000 7f4de759e700 1 -- start start 2023-11-28T11:08:56.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 -- 172.21.15.83:0/3393739356 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 msgr2=0x7f2fa4160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/3393739356 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 0x7f2fa4160bb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009e50 tx=0x7f2f9c00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 -- 172.21.15.83:0/3393739356 shutdown_connections 2023-11-28T11:08:56.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/3393739356 >> v2:172.21.15.83:3300/0 conn(0x7f2fa40d0d10 0x7f2fa40d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/3393739356 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa40b4980 0x7f2fa4160bb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 -- 172.21.15.83:0/3393739356 >> 172.21.15.83:0/3393739356 conn(0x7f2fa406ced0 msgr2=0x7f2fa40cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 -- 172.21.15.83:0/3393739356 shutdown_connections 2023-11-28T11:08:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.661+0000 7f4de759e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 0x7f4de8085730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.661+0000 7f4de759e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80710b0 0x7f4de8160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.661+0000 7f4de759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de80ccb70 0x7f4de8160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.661+0000 7f4de759e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de816be70 con 0x7f4de80ccb70 2023-11-28T11:08:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.661+0000 7f4de759e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de816c150 con 0x7f4de806b7b0 2023-11-28T11:08:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.661+0000 7f4de759e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8165760 con 0x7f4de80710b0 2023-11-28T11:08:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.660+0000 7f2fac6f7700 1 -- 172.21.15.83:0/3393739356 wait complete. 2023-11-28T11:08:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.662+0000 7f2fac6f7700 1 Processor -- start 2023-11-28T11:08:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.662+0000 7f45baef6700 1 -- 172.21.15.83:0/3916263157 shutdown_connections 2023-11-28T11:08:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.662+0000 7f45baef6700 1 -- 172.21.15.83:0/3916263157 wait complete. 2023-11-28T11:08:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 0x7f4de8085730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de5d9b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80710b0 0x7f4de8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 0x7f4de8085730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:36048/0 (socket says 172.21.15.83:36048) 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 -- 172.21.15.83:0/3438885490 learned_addr learned my addr 172.21.15.83:0/3438885490 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 -- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80710b0 msgr2=0x7f4de8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80710b0 0x7f4de8160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 -- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de80ccb70 msgr2=0x7f4de8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de80ccb70 0x7f4de8160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 -- 172.21.15.83:0/3438885490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de000bc80 con 0x7f4de806b7b0 2023-11-28T11:08:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.663+0000 7f4de659c700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 0x7f4de8085730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800a780 tx=0x7f4dd800ab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.662+0000 7f2fac6f7700 1 -- start start 2023-11-28T11:08:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fac6f7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa406b3a0 0x7f2fa4157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fac6f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa40b4980 0x7f2fa40c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fac6f7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 0x7f2fa41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fac6f7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa40c9bb0 con 0x7f2fa40d0d10 2023-11-28T11:08:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fac6f7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa40c96e0 con 0x7f2fa40b4980 2023-11-28T11:08:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fac6f7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41562d0 con 0x7f2fa406b3a0 2023-11-28T11:08:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 0x7f2fa41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 0x7f2fa41522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49062/0 (socket says 172.21.15.83:49062) 2023-11-28T11:08:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 -- 172.21.15.83:0/2581843874 learned_addr learned my addr 172.21.15.83:0/2581843874 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2fa9c92700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa40b4980 0x7f2fa40c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800f040 con 0x7f4de806b7b0 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f45baef6700 1 Processor -- start 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.665+0000 7f45baef6700 1 -- start start 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45baef6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b41ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45baef6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b41651e0 0x7f45b40c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45baef6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b4159ce0 0x7f45b40ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45baef6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4159a00 con 0x7f45b406acd0 2023-11-28T11:08:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45baef6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b41ea5a0 con 0x7f45b4159ce0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45baef6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40b0560 con 0x7f45b41651e0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45b8c92700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b41ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45b8c92700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b41ecc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49064/0 (socket says 172.21.15.83:49064) 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45b9493700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b4159ce0 0x7f45b40ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.666+0000 7f45b8c92700 1 -- 172.21.15.83:0/1123352371 learned_addr learned my addr 172.21.15.83:0/1123352371 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd800ea30 con 0x7f4de806b7b0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800eda0 con 0x7f4de806b7b0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de806caa0 con 0x7f4de806b7b0 2023-11-28T11:08:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de816dd30 con 0x7f4de806b7b0 2023-11-28T11:08:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 -- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa406b3a0 msgr2=0x7f2fa4157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa406b3a0 0x7f2fa4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 -- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa40b4980 msgr2=0x7f2fa40c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa40b4980 0x7f2fa40c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 -- 172.21.15.83:0/2581843874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c009b30 con 0x7f2fa40d0d10 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.664+0000 7f2faac94700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 0x7f2fa41522a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400b550 tx=0x7f2f9400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.665+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94011830 con 0x7f2fa40d0d10 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.665+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f940119b0 con 0x7f2fa40d0d10 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.665+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94011d60 con 0x7f2fa40d0d10 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.665+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa40cdb40 con 0x7f2fa40d0d10 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.665+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa40cd860 con 0x7f2fa40d0d10 2023-11-28T11:08:56.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b9493700 1 -- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b41651e0 msgr2=0x7f45b40c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b9493700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b41651e0 0x7f45b40c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b9493700 1 -- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 msgr2=0x7f45b41ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b9493700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b41ecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b9493700 1 -- 172.21.15.83:0/1123352371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4009550 con 0x7f45b4159ce0 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b8c92700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b41ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.667+0000 7f45b9493700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b4159ce0 0x7f45b40ce110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ca70 tx=0x7f45ac00cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.668+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00a980 con 0x7f45b4159ce0 2023-11-28T11:08:56.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.668+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac00aae0 con 0x7f45b4159ce0 2023-11-28T11:08:56.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.668+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00ae50 con 0x7f45b4159ce0 2023-11-28T11:08:56.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.668+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b406ba90 con 0x7f45b4159ce0 2023-11-28T11:08:56.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.668+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b406a270 con 0x7f45b4159ce0 2023-11-28T11:08:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.669+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f94021070 con 0x7f2fa40d0d10 2023-11-28T11:08:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.669+0000 7f2f9affd700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f90068520 0x7f2f9006a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2f94085ea0 con 0x7f2fa40d0d10 2023-11-28T11:08:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4dd8016480 con 0x7f4de806b7b0 2023-11-28T11:08:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f88005320 con 0x7f2fa40d0d10 2023-11-28T11:08:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f4dd6ffd700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4dc80683f0 0x7f4dc806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4dd8086a50 con 0x7f4de806b7b0 2023-11-28T11:08:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45ac019030 con 0x7f45b4159ce0 2023-11-28T11:08:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.670+0000 7f45b17fa700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45a0068340 0x7f45a006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.671+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f45ac0859b0 con 0x7f45b4159ce0 2023-11-28T11:08:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.671+0000 7f4de5d9b700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4dc80683f0 0x7f4dc806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.671+0000 7f4de5d9b700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4dc80683f0 0x7f4dc806a8a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4de8165eb0 tx=0x7f4ddc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.672+0000 7f45b8c92700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45a0068340 0x7f45a006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.672+0000 7f2faa493700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f90068520 0x7f2f9006a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.672+0000 7f2faa493700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f90068520 0x7f2f9006a9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2fa415bfa0 tx=0x7f2f9c010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.672+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc0052f0 con 0x7f4de806b7b0 2023-11-28T11:08:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45940052f0 con 0x7f45b4159ce0 2023-11-28T11:08:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f45b8c92700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45a0068340 0x7f45a006a7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009130 tx=0x7f45a4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.673+0000 7f7a7dbdc700 1 Processor -- start 2023-11-28T11:08:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.673+0000 7f7a7dbdc700 1 -- start start 2023-11-28T11:08:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a7dbdc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a78071330 0x7f7a7816cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a7dbdc700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7a780cc750 0x7f7a780ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a7dbdc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780cd0e0 con 0x7f7a780899e0 2023-11-28T11:08:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a7dbdc700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7816de80 con 0x7f7a780cc750 2023-11-28T11:08:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a7dbdc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78169860 con 0x7f7a78071330 2023-11-28T11:08:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a76ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7a780899e0 0x7f7a780c91d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46938/0 (socket says 172.21.15.83:46938) 2023-11-28T11:08:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.674+0000 7f7a76ffd700 1 -- 172.21.15.83:0/1404583135 learned_addr learned my addr 172.21.15.83:0/1404583135 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.675+0000 7f7a777fe700 1 --2- 172.21.15.83:0/1404583135 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a78071330 0x7f7a7816cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a777fe700 1 -- 172.21.15.83:0/1404583135 >> v2:172.21.15.90:3300/0 conn(0x7f7a780cc750 msgr2=0x7f7a780ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a777fe700 1 --2- 172.21.15.83:0/1404583135 >> v2:172.21.15.90:3300/0 conn(0x7f7a780cc750 0x7f7a780ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a777fe700 1 -- 172.21.15.83:0/1404583135 >> v1:172.21.15.83:6789/0 conn(0x7f7a780899e0 legacy=0x7f7a780c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a777fe700 1 -- 172.21.15.83:0/1404583135 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78071c00 con 0x7f7a78071330 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a777fe700 1 --2- 172.21.15.83:0/1404583135 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a78071330 0x7f7a7816cf50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007830 tx=0x7f7a68007b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfdfb02c2dd88b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a767fc700 1 -- 172.21.15.83:0/1404583135 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800f070 con 0x7f7a78071330 2023-11-28T11:08:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a767fc700 1 -- 172.21.15.83:0/1404583135 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800c890 con 0x7f7a78071330 2023-11-28T11:08:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.676+0000 7f7a767fc700 1 -- 172.21.15.83:0/1404583135 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800cc20 con 0x7f7a78071330 2023-11-28T11:08:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.677+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/1404583135 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a78071330 msgr2=0x7f7a7816cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.677+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/1404583135 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a78071330 0x7f7a7816cf50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007830 tx=0x7f7a68007b40 comp rx=0 tx=0).stop 2023-11-28T11:08:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.678+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4dd8052ef0 con 0x7f4de806b7b0 2023-11-28T11:08:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/1404583135 shutdown_connections 2023-11-28T11:08:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/1404583135 >> v2:172.21.15.90:3300/0 conn(0x7f7a780cc750 0x7f7a780ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/1404583135 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a78071330 0x7f7a7816cf50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/1404583135 >> 172.21.15.83:0/1404583135 conn(0x7f7a780b1380 msgr2=0x7f7a780ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f45ac04b5e0 con 0x7f45b4159ce0 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/1404583135 shutdown_connections 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/1404583135 wait complete. 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.681+0000 7f7a7dbdc700 1 Processor -- start 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.681+0000 7f7a7dbdc700 1 -- start start 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f7a7dbdc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a78071330 0x7f7a780b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f7a7dbdc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a780899e0 0x7f7a782020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f7a7dbdc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 0x7f7a780c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f7a7dbdc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780bace0 con 0x7f7a78071330 2023-11-28T11:08:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f7a7dbdc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780b4780 con 0x7f7a780899e0 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.682+0000 7f7a7dbdc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c5cf0 con 0x7f7a780cc750 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 0x7f7a780c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 0x7f7a780c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:33286/0 (socket says 172.21.15.83:33286) 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 -- 172.21.15.83:0/3158813052 learned_addr learned my addr 172.21.15.83:0/3158813052 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a76ffd700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a780899e0 0x7f7a782020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 -- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a780899e0 msgr2=0x7f7a782020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a780899e0 0x7f7a782020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 -- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a78071330 msgr2=0x7f7a780b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a78071330 0x7f7a780b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 -- 172.21.15.83:0/3158813052 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68007510 con 0x7f7a780cc750 2023-11-28T11:08:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.683+0000 7f7a77fff700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 0x7f7a780c94a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000db90 tx=0x7f7a7000dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.679+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f940523a0 con 0x7f2fa40d0d10 2023-11-28T11:08:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.686+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000e750 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.686+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7000e8d0 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.686+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000ec80 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.686+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7816d8c0 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.686+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7816d5e0 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.688+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a7001c020 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.688+0000 7f7a5ffff700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a6406c750 0x7f7a6406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.688+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a70006e40 con 0x7f7a780cc750 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.689+0000 7f7a777fe700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a6406c750 0x7f7a6406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.689+0000 7f7a777fe700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a6406c750 0x7f7a6406ec00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007b80 tx=0x7f7a68011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.692+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a580052f0 con 0x7f7a780cc750 2023-11-28T11:08:56.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.697+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a70052270 con 0x7f7a780cc750 2023-11-28T11:08:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.762+0000 7f2ebcbb4700 1 Processor -- start 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.763+0000 7f2ebcbb4700 1 -- start start 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.764+0000 7f2ebcbb4700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2eb80cf290 0x7f2eb806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.764+0000 7f2ebcbb4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2eb8158700 0x7f2eb8070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.764+0000 7f2ebcbb4700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb806d7d0 con 0x7f2eb815cd90 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.764+0000 7f2ebcbb4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb806cea0 con 0x7f2eb8158700 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.764+0000 7f2ebcbb4700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb806d180 con 0x7f2eb80cf290 2023-11-28T11:08:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.765+0000 7f2eb6d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2eb8158700 0x7f2eb8070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.765+0000 7f2eb6d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2eb8158700 0x7f2eb8070ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49090/0 (socket says 172.21.15.83:49090) 2023-11-28T11:08:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.765+0000 7f2eb6d9d700 1 -- 172.21.15.83:0/2340680519 learned_addr learned my addr 172.21.15.83:0/2340680519 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.765+0000 7f2eb659c700 1 --2- 172.21.15.83:0/2340680519 >> v2:172.21.15.90:3300/0 conn(0x7f2eb80cf290 0x7f2eb806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.766+0000 7f2eb6d9d700 1 -- 172.21.15.83:0/2340680519 >> v2:172.21.15.90:3300/0 conn(0x7f2eb80cf290 msgr2=0x7f2eb806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.766+0000 7f2eb6d9d700 1 --2- 172.21.15.83:0/2340680519 >> v2:172.21.15.90:3300/0 conn(0x7f2eb80cf290 0x7f2eb806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.766+0000 7f2eb6d9d700 1 -- 172.21.15.83:0/2340680519 >> v1:172.21.15.90:6789/0 conn(0x7f2eb815cd90 legacy=0x7f2eb806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.766+0000 7f2eb6d9d700 1 -- 172.21.15.83:0/2340680519 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb806d460 con 0x7f2eb8158700 2023-11-28T11:08:56.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.766+0000 7f2eb6d9d700 1 --2- 172.21.15.83:0/2340680519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb8158700 0x7f2eb8070ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806bd80 tx=0x7f2eb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c624cf1e1fe710 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.767+0000 7f2eb559a700 1 -- 172.21.15.83:0/2340680519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000e070 con 0x7f2eb8158700 2023-11-28T11:08:56.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.767+0000 7f2eb559a700 1 -- 172.21.15.83:0/2340680519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb0003a20 con 0x7f2eb8158700 2023-11-28T11:08:56.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.767+0000 7f2eb559a700 1 -- 172.21.15.83:0/2340680519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0003db0 con 0x7f2eb8158700 2023-11-28T11:08:56.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.768+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/2340680519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb8158700 msgr2=0x7f2eb8070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.768+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/2340680519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb8158700 0x7f2eb8070ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806bd80 tx=0x7f2eb000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.769+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/2340680519 shutdown_connections 2023-11-28T11:08:56.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.769+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/2340680519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb8158700 0x7f2eb8070ad0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.769+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/2340680519 >> v2:172.21.15.90:3300/0 conn(0x7f2eb80cf290 0x7f2eb806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.769+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/2340680519 >> 172.21.15.83:0/2340680519 conn(0x7f2eb80b1380 msgr2=0x7f2eb80ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.769+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/2340680519 shutdown_connections 2023-11-28T11:08:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.769+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/2340680519 wait complete. 2023-11-28T11:08:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.771+0000 7f2ebcbb4700 1 Processor -- start 2023-11-28T11:08:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.771+0000 7f2ebcbb4700 1 -- start start 2023-11-28T11:08:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2ebcbb4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2eb80cf290 0x7f2eb80c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2ebcbb4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eb8158700 0x7f2eb80c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2ebcbb4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb815cd90 0x7f2eb80c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2ebcbb4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c51c0 con 0x7f2eb815cd90 2023-11-28T11:08:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2ebcbb4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c6350 con 0x7f2eb8158700 2023-11-28T11:08:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2ebcbb4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c6630 con 0x7f2eb80cf290 2023-11-28T11:08:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2eb5d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eb8158700 0x7f2eb80c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2eb5d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eb8158700 0x7f2eb80c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:36092/0 (socket says 172.21.15.83:36092) 2023-11-28T11:08:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.773+0000 7f2eb5d9b700 1 -- 172.21.15.83:0/292397263 learned_addr learned my addr 172.21.15.83:0/292397263 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.774+0000 7f2eb659c700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2eb80cf290 0x7f2eb80c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.775+0000 7f2eb659c700 1 -- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eb8158700 msgr2=0x7f2eb80c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.775+0000 7f2eb659c700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eb8158700 0x7f2eb80c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.775+0000 7f2eb659c700 1 -- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb815cd90 msgr2=0x7f2eb80c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.775+0000 7f2eb659c700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb815cd90 0x7f2eb80c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.775+0000 7f2eb659c700 1 -- 172.21.15.83:0/292397263 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0009b30 con 0x7f2eb80cf290 2023-11-28T11:08:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.776+0000 7f2eb659c700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2eb80cf290 0x7f2eb80c89b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e900 tx=0x7f2eac00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.776+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00ee50 con 0x7f2eb80cf290 2023-11-28T11:08:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.776+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac00c850 con 0x7f2eb80cf290 2023-11-28T11:08:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.776+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00cbc0 con 0x7f2eb80cf290 2023-11-28T11:08:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.776+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb8158b80 con 0x7f2eb80cf290 2023-11-28T11:08:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.776+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb815d1e0 con 0x7f2eb80cf290 2023-11-28T11:08:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.779+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c0052f0 con 0x7f2eb80cf290 2023-11-28T11:08:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.781+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2eac019030 con 0x7f2eb80cf290 2023-11-28T11:08:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.781+0000 7f2ea6ffd700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e980683c0 0x7f2e9806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.781+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2eac085ec0 con 0x7f2eb80cf290 2023-11-28T11:08:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.782+0000 7f2eb5d9b700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e980683c0 0x7f2e9806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.782+0000 7f2eb5d9b700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e980683c0 0x7f2e9806a870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80c8900 tx=0x7f2ea8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.785+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2eac0523c0 con 0x7f2eb80cf290 2023-11-28T11:08:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.910+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f59c40055a0 con 0x7f59e4158700 2023-11-28T11:08:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.919+0000 7f59e17fa700 1 -- 172.21.15.83:0/2569239941 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59d4053440 con 0x7f59e4158700 2023-11-28T11:08:56.925 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:56.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59d0068340 msgr2=0x7f59d006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59d0068340 0x7f59d006a7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59e40b01b0 tx=0x7f59dc0015f0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 msgr2=0x7f59e41f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 0x7f59e41f8630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f59d40090b0 tx=0x7f59d4004750 comp rx=0 tx=0).stop 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 shutdown_connections 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59d0068340 0x7f59d006a7f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4158700 0x7f59e41f8630 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59e406e440 0x7f59e40c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 --2- 172.21.15.83:0/2569239941 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59e40cf290 0x7f59e40b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.925+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 >> 172.21.15.83:0/2569239941 conn(0x7f59e40b1380 msgr2=0x7f59e40ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.926+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 shutdown_connections 2023-11-28T11:08:56.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.926+0000 7f59eaab1700 1 -- 172.21.15.83:0/2569239941 wait complete. 2023-11-28T11:08:56.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-11-28T11:08:56.945 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:56.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.979+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f30101efe30 con 0x7f3010158790 2023-11-28T11:08:56.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.979+0000 7f2ff77fe700 1 -- 172.21.15.83:0/1715247399 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30101efe30 con 0x7f3010158790 2023-11-28T11:08:56.981 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2ffc068340 msgr2=0x7f2ffc06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2ffc068340 0x7f2ffc06a7f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f300400b240 tx=0x7f300400b420 comp rx=0 tx=0).stop 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 msgr2=0x7f30100c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 0x7f30100c8260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f300000bc60 tx=0x7f3000009e20 comp rx=0 tx=0).stop 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 shutdown_connections 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f301015cd90 0x7f30100c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3010070ad0 0x7f30101ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2ffc068340 0x7f2ffc06a7f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 --2- 172.21.15.83:0/1715247399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3010158790 0x7f30100c8260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.984+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 >> 172.21.15.83:0/1715247399 conn(0x7f30100b1380 msgr2=0x7f3010154440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.985+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f11a00055a0 con 0x7f11bc070ad0 2023-11-28T11:08:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.986+0000 7f11b3fff700 1 -- 172.21.15.83:0/1946636579 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11b4019070 con 0x7f11bc070ad0 2023-11-28T11:08:56.989 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-28T11:08:56.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.991+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 shutdown_connections 2023-11-28T11:08:56.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.991+0000 7f3015355700 1 -- 172.21.15.83:0/1715247399 wait complete. 2023-11-28T11:08:56.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.991+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f119c0683c0 msgr2=0x7f119c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.991+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f119c0683c0 0x7f119c06a870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0c5000 tx=0x7f11ac009210 comp rx=0 tx=0).stop 2023-11-28T11:08:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.991+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11bc070ad0 msgr2=0x7f11bc150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.991+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11bc070ad0 0x7f11bc150110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11b400be20 tx=0x7f11b4009f70 comp rx=0 tx=0).stop 2023-11-28T11:08:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.995+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 shutdown_connections 2023-11-28T11:08:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.995+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f11bc15cd90 0x7f11bc150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.995+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11bc070ad0 0x7f11bc150110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.995+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f119c0683c0 0x7f119c06a870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.995+0000 7f11c1a97700 1 --2- 172.21.15.83:0/1946636579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11bc158790 0x7f11bc0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.995+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 >> 172.21.15.83:0/1946636579 conn(0x7f11bc0b1380 msgr2=0x7f11bc0ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.997+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 shutdown_connections 2023-11-28T11:08:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:56.997+0000 7f11c1a97700 1 -- 172.21.15.83:0/1946636579 wait complete. 2023-11-28T11:08:57.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-11-28T11:08:57.008 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.008+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f880055d0 con 0x7f2fa40d0d10 2023-11-28T11:08:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.009+0000 7f2f9affd700 1 -- 172.21.15.83:0/2581843874 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f94015020 con 0x7f2fa40d0d10 2023-11-28T11:08:57.011 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.012+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f90068520 msgr2=0x7f2f9006a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.012+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f90068520 0x7f2f9006a9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2fa415bfa0 tx=0x7f2f9c010000 comp rx=0 tx=0).stop 2023-11-28T11:08:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.012+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 msgr2=0x7f2fa41522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.012+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 0x7f2fa41522a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400b550 tx=0x7f2f9400b860 comp rx=0 tx=0).stop 2023-11-28T11:08:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 shutdown_connections 2023-11-28T11:08:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f90068520 0x7f2f9006a9d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa40d0d10 0x7f2fa41522a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa406b3a0 0x7f2fa4157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 --2- 172.21.15.83:0/2581843874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa40b4980 0x7f2fa40c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 >> 172.21.15.83:0/2581843874 conn(0x7f2fa406ced0 msgr2=0x7f2fa41603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 shutdown_connections 2023-11-28T11:08:57.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.013+0000 7f2fac6f7700 1 -- 172.21.15.83:0/2581843874 wait complete. 2023-11-28T11:08:57.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-11-28T11:08:57.016 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-11-28T11:08:57.027 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.064+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4dcc0055a0 con 0x7f4de806b7b0 2023-11-28T11:08:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.065+0000 7f4dd6ffd700 1 -- 172.21.15.83:0/3438885490 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4dd8056510 con 0x7f4de806b7b0 2023-11-28T11:08:57.066 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7a580055a0 con 0x7f7a780cc750 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4dc80683f0 msgr2=0x7f4dc806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4dc80683f0 0x7f4dc806a8a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4de8165eb0 tx=0x7f4ddc009040 comp rx=0 tx=0).stop 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 msgr2=0x7f4de8085730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 0x7f4de8085730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800a780 tx=0x7f4dd800ab40 comp rx=0 tx=0).stop 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 shutdown_connections 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4dc80683f0 0x7f4dc806a8a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de80ccb70 0x7f4de8160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de806b7b0 0x7f4de8085730 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 --2- 172.21.15.83:0/3438885490 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4de80710b0 0x7f4de8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 >> 172.21.15.83:0/3438885490 conn(0x7f4de80cea00 msgr2=0x7f4de80708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 shutdown_connections 2023-11-28T11:08:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f4de759e700 1 -- 172.21.15.83:0/3438885490 wait complete. 2023-11-28T11:08:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.068+0000 7f7a5ffff700 1 -- 172.21.15.83:0/3158813052 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a70055890 con 0x7f7a780cc750 2023-11-28T11:08:57.071 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2023-11-28T11:08:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f45940055a0 con 0x7f45b4159ce0 2023-11-28T11:08:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a6406c750 msgr2=0x7f7a6406ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a6406c750 0x7f7a6406ec00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007b80 tx=0x7f7a68011000 comp rx=0 tx=0).stop 2023-11-28T11:08:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 msgr2=0x7f7a780c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 0x7f7a780c94a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000db90 tx=0x7f7a7000dea0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 shutdown_connections 2023-11-28T11:08:57.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a780cc750 0x7f7a780c94a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a6406c750 0x7f7a6406ec00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a78071330 0x7f7a780b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 --2- 172.21.15.83:0/3158813052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a780899e0 0x7f7a782020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 >> 172.21.15.83:0/3158813052 conn(0x7f7a780b1380 msgr2=0x7f7a780afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 shutdown_connections 2023-11-28T11:08:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.071+0000 7f7a7dbdc700 1 -- 172.21.15.83:0/3158813052 wait complete. 2023-11-28T11:08:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.072+0000 7f45b17fa700 1 -- 172.21.15.83:0/1123352371 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45ac017030 con 0x7f45b4159ce0 2023-11-28T11:08:57.076 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:57.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.074+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45a0068340 msgr2=0x7f45a006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.074+0000 7f45baef6700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45a0068340 0x7f45a006a7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009130 tx=0x7f45a4008030 comp rx=0 tx=0).stop 2023-11-28T11:08:57.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.074+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b4159ce0 msgr2=0x7f45b40ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.074+0000 7f45baef6700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b4159ce0 0x7f45b40ce110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ca70 tx=0x7f45ac00cd80 comp rx=0 tx=0).stop 2023-11-28T11:08:57.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.075+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 shutdown_connections 2023-11-28T11:08:57.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.075+0000 7f45baef6700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b4159ce0 0x7f45b40ce110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.075+0000 7f45baef6700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45a0068340 0x7f45a006a7f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.075+0000 7f45baef6700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b406acd0 0x7f45b41ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.075+0000 7f45baef6700 1 --2- 172.21.15.83:0/1123352371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f45b41651e0 0x7f45b40c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.075+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 >> 172.21.15.83:0/1123352371 conn(0x7f45b40cba50 msgr2=0x7f45b406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.076+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 shutdown_connections 2023-11-28T11:08:57.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.076+0000 7f45baef6700 1 -- 172.21.15.83:0/1123352371 wait complete. 2023-11-28T11:08:57.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-11-28T11:08:57.086 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-11-28T11:08:57.091 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-11-28T11:08:57.093 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.129+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2e9c005870 con 0x7f2eb80cf290 2023-11-28T11:08:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.130+0000 7f2ea6ffd700 1 -- 172.21.15.83:0/292397263 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2eac0559e0 con 0x7f2eb80cf290 2023-11-28T11:08:57.132 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e980683c0 msgr2=0x7f2e9806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e980683c0 0x7f2e9806a870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80c8900 tx=0x7f2ea8009190 comp rx=0 tx=0).stop 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2eb80cf290 msgr2=0x7f2eb80c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2eb80cf290 0x7f2eb80c89b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e900 tx=0x7f2eac00ec10 comp rx=0 tx=0).stop 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 shutdown_connections 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e980683c0 0x7f2e9806a870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eb815cd90 0x7f2eb80c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2eb80cf290 0x7f2eb80c89b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 --2- 172.21.15.83:0/292397263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2eb8158700 0x7f2eb80c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 >> 172.21.15.83:0/292397263 conn(0x7f2eb80b1380 msgr2=0x7f2eb80ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 shutdown_connections 2023-11-28T11:08:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.132+0000 7f2ebcbb4700 1 -- 172.21.15.83:0/292397263 wait complete. 2023-11-28T11:08:57.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-11-28T11:08:57.146 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:57.146 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:08:57.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.305+0000 7f4637066700 1 Processor -- start 2023-11-28T11:08:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.305+0000 7f4637066700 1 -- start start 2023-11-28T11:08:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.306+0000 7f4637066700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f46300cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.306+0000 7f4637066700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f46300b18a0 0x7f46300b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.306+0000 7f4637066700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463006ca60 con 0x7f463006c5f0 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.306+0000 7f4637066700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463015d480 con 0x7f46300b18a0 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.306+0000 7f4637066700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300b2260 con 0x7f4630159830 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4634e02700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f46300cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4635603700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f46300b18a0 0x7f46300b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462ffff700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f463006c5f0 0x7f463015de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45310/0 (socket says 172.21.15.83:45310) 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462ffff700 1 -- 172.21.15.83:0/1387055611 learned_addr learned my addr 172.21.15.83:0/1387055611 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462f7fe700 1 -- 172.21.15.83:0/1387055611 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153029017 0 0) 0x7f463006ca60 con 0x7f463006c5f0 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462f7fe700 1 -- 172.21.15.83:0/1387055611 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46200035b0 con 0x7f463006c5f0 2023-11-28T11:08:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4635603700 1 -- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 msgr2=0x7f46300cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4635603700 1 --2- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f46300cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4635603700 1 -- 172.21.15.83:0/1387055611 >> v1:172.21.15.90:6789/0 conn(0x7f463006c5f0 legacy=0x7f463015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4635603700 1 -- 172.21.15.83:0/1387055611 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46300cf790 con 0x7f46300b18a0 2023-11-28T11:08:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4634e02700 1 --2- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f46300cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f4635603700 1 --2- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f46300b1d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4628009d90 tx=0x7f462800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40139c758ebca2ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462f7fe700 1 -- 172.21.15.83:0/1387055611 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800e070 con 0x7f46300b18a0 2023-11-28T11:08:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462f7fe700 1 -- 172.21.15.83:0/1387055611 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462800bb20 con 0x7f46300b18a0 2023-11-28T11:08:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.307+0000 7f462f7fe700 1 -- 172.21.15.83:0/1387055611 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800be90 con 0x7f46300b18a0 2023-11-28T11:08:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 -- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 msgr2=0x7f46300b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 --2- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f46300b1d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4628009d90 tx=0x7f462800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 -- 172.21.15.83:0/1387055611 shutdown_connections 2023-11-28T11:08:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 --2- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f46300b1d20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 --2- 172.21.15.83:0/1387055611 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f46300cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 -- 172.21.15.83:0/1387055611 >> 172.21.15.83:0/1387055611 conn(0x7f463006e9a0 msgr2=0x7f46300cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 -- 172.21.15.83:0/1387055611 shutdown_connections 2023-11-28T11:08:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.308+0000 7f4637066700 1 -- 172.21.15.83:0/1387055611 wait complete. 2023-11-28T11:08:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.309+0000 7f4637066700 1 Processor -- start 2023-11-28T11:08:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.309+0000 7f4637066700 1 -- start start 2023-11-28T11:08:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4637066700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 0x7f46301f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4637066700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f4630157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4637066700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f4630157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4637066700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463015ab70 con 0x7f463006c5f0 2023-11-28T11:08:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4637066700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630157d90 con 0x7f46300b18a0 2023-11-28T11:08:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4637066700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300b47d0 con 0x7f4630159830 2023-11-28T11:08:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4634e02700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 0x7f46301f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4634e02700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 0x7f46301f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49110/0 (socket says 172.21.15.83:49110) 2023-11-28T11:08:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.310+0000 7f4634e02700 1 -- 172.21.15.83:0/2258370630 learned_addr learned my addr 172.21.15.83:0/2258370630 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f462ffff700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f4630157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4635603700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f4630157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4634e02700 1 -- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 msgr2=0x7f4630157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4634e02700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f4630157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4634e02700 1 -- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 msgr2=0x7f4630157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4634e02700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f4630157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4634e02700 1 -- 172.21.15.83:0/2258370630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240094a0 con 0x7f463006c5f0 2023-11-28T11:08:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4634e02700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 0x7f46301f4ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f462400ebe0 tx=0x7f4624009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400c910 con 0x7f463006c5f0 2023-11-28T11:08:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400ca70 con 0x7f463006c5f0 2023-11-28T11:08:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4628009a70 con 0x7f463006c5f0 2023-11-28T11:08:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400cde0 con 0x7f463006c5f0 2023-11-28T11:08:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.311+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463015d690 con 0x7f463006c5f0 2023-11-28T11:08:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.312+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46300cfa70 con 0x7f463006c5f0 2023-11-28T11:08:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.312+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4624019030 con 0x7f463006c5f0 2023-11-28T11:08:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.313+0000 7f462d7fa700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46140682f0 0x7f461406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.313+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4624086010 con 0x7f463006c5f0 2023-11-28T11:08:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.313+0000 7f462ffff700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46140682f0 0x7f461406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.313+0000 7f462ffff700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46140682f0 0x7f461406a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f461c005910 tx=0x7f461c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.317+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4624052580 con 0x7f463006c5f0 2023-11-28T11:08:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.493+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f463015a3b0 con 0x7f46140682f0 2023-11-28T11:08:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.503+0000 7f462d7fa700 1 -- 172.21.15.83:0/2258370630 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367512 (secure 0 0 0) 0x7f463015a3b0 con 0x7f46140682f0 2023-11-28T11:08:57.509 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46140682f0 msgr2=0x7f461406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46140682f0 0x7f461406a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f461c005910 tx=0x7f461c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 msgr2=0x7f46301f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 0x7f46301f4ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f462400ebe0 tx=0x7f4624009ec0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 shutdown_connections 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4630159830 0x7f4630157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46140682f0 0x7f461406a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463006c5f0 0x7f46301f4ef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 --2- 172.21.15.83:0/2258370630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46300b18a0 0x7f4630157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 >> 172.21.15.83:0/2258370630 conn(0x7f463006e9a0 msgr2=0x7f46300b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 shutdown_connections 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.507+0000 7f4637066700 1 -- 172.21.15.83:0/2258370630 wait complete. 2023-11-28T11:08:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:08:57.520 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T11:08:55.572857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67136,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907584,"statfs":{"total":773094113280,"available":773025366016,"internally_reserved":0,"allocated":3276800,"data_stored":2024910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002505"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542681+0000","last_change":"2023-11-28T11:08:35.535687+0000","last_active":"2023-11-28T11:08:39.542681+0000","last_peered":"2023-11-28T11:08:39.542681+0000","last_clean":"2023-11-28T11:08:39.542681+0000","last_became_active":"2023-11-28T11:08:35.535025+0000","last_became_peered":"2023-11-28T11:08:35.535025+0000","last_unstale":"2023-11-28T11:08:39.542681+0000","last_undegraded":"2023-11-28T11:08:39.542681+0000","last_fullsized":"2023-11-28T11:08:39.542681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:21.578670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546994+0000","last_change":"2023-11-28T11:08:35.534288+0000","last_active":"2023-11-28T11:08:39.546994+0000","last_peered":"2023-11-28T11:08:39.546994+0000","last_clean":"2023-11-28T11:08:39.546994+0000","last_became_active":"2023-11-28T11:08:35.533997+0000","last_became_peered":"2023-11-28T11:08:35.533997+0000","last_unstale":"2023-11-28T11:08:39.546994+0000","last_undegraded":"2023-11-28T11:08:39.546994+0000","last_fullsized":"2023-11-28T11:08:39.546994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556945+0000","last_change":"2023-11-28T11:08:35.553042+0000","last_active":"2023-11-28T11:08:39.556945+0000","last_peered":"2023-11-28T11:08:39.556945+0000","last_clean":"2023-11-28T11:08:39.556945+0000","last_became_active":"2023-11-28T11:08:35.552693+0000","last_became_peered":"2023-11-28T11:08:35.552693+0000","last_unstale":"2023-11-28T11:08:39.556945+0000","last_undegraded":"2023-11-28T11:08:39.556945+0000","last_fullsized":"2023-11-28T11:08:39.556945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:43.204644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551967+0000","last_change":"2023-11-28T11:08:35.538585+0000","last_active":"2023-11-28T11:08:39.551967+0000","last_peered":"2023-11-28T11:08:39.551967+0000","last_clean":"2023-11-28T11:08:39.551967+0000","last_became_active":"2023-11-28T11:08:35.538074+0000","last_became_peered":"2023-11-28T11:08:35.538074+0000","last_unstale":"2023-11-28T11:08:39.551967+0000","last_undegraded":"2023-11-28T11:08:39.551967+0000","last_fullsized":"2023-11-28T11:08:39.551967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:13.796245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556389+0000","last_change":"2023-11-28T11:08:35.548649+0000","last_active":"2023-11-28T11:08:39.556389+0000","last_peered":"2023-11-28T11:08:39.556389+0000","last_clean":"2023-11-28T11:08:39.556389+0000","last_became_active":"2023-11-28T11:08:35.546646+0000","last_became_peered":"2023-11-28T11:08:35.546646+0000","last_unstale":"2023-11-28T11:08:39.556389+0000","last_undegraded":"2023-11-28T11:08:39.556389+0000","last_fullsized":"2023-11-28T11:08:39.556389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556449+0000","last_change":"2023-11-28T11:08:35.545320+0000","last_active":"2023-11-28T11:08:39.556449+0000","last_peered":"2023-11-28T11:08:39.556449+0000","last_clean":"2023-11-28T11:08:39.556449+0000","last_became_active":"2023-11-28T11:08:35.545051+0000","last_became_peered":"2023-11-28T11:08:35.545051+0000","last_unstale":"2023-11-28T11:08:39.556449+0000","last_undegraded":"2023-11-28T11:08:39.556449+0000","last_fullsized":"2023-11-28T11:08:39.556449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549589+0000","last_change":"2023-11-28T11:08:35.543641+0000","last_active":"2023-11-28T11:08:39.549589+0000","last_peered":"2023-11-28T11:08:39.549589+0000","last_clean":"2023-11-28T11:08:39.549589+0000","last_became_active":"2023-11-28T11:08:35.543407+0000","last_became_peered":"2023-11-28T11:08:35.543407+0000","last_unstale":"2023-11-28T11:08:39.549589+0000","last_undegraded":"2023-11-28T11:08:39.549589+0000","last_fullsized":"2023-11-28T11:08:39.549589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546097+0000","last_change":"2023-11-28T11:08:35.538922+0000","last_active":"2023-11-28T11:08:39.546097+0000","last_peered":"2023-11-28T11:08:39.546097+0000","last_clean":"2023-11-28T11:08:39.546097+0000","last_became_active":"2023-11-28T11:08:35.538455+0000","last_became_peered":"2023-11-28T11:08:35.538455+0000","last_unstale":"2023-11-28T11:08:39.546097+0000","last_undegraded":"2023-11-28T11:08:39.546097+0000","last_fullsized":"2023-11-28T11:08:39.546097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549757+0000","last_change":"2023-11-28T11:08:35.550345+0000","last_active":"2023-11-28T11:08:39.549757+0000","last_peered":"2023-11-28T11:08:39.549757+0000","last_clean":"2023-11-28T11:08:39.549757+0000","last_became_active":"2023-11-28T11:08:35.546222+0000","last_became_peered":"2023-11-28T11:08:35.546222+0000","last_unstale":"2023-11-28T11:08:39.549757+0000","last_undegraded":"2023-11-28T11:08:39.549757+0000","last_fullsized":"2023-11-28T11:08:39.549757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546977+0000","last_change":"2023-11-28T11:08:35.550152+0000","last_active":"2023-11-28T11:08:39.546977+0000","last_peered":"2023-11-28T11:08:39.546977+0000","last_clean":"2023-11-28T11:08:39.546977+0000","last_became_active":"2023-11-28T11:08:35.549618+0000","last_became_peered":"2023-11-28T11:08:35.549618+0000","last_unstale":"2023-11-28T11:08:39.546977+0000","last_undegraded":"2023-11-28T11:08:39.546977+0000","last_fullsized":"2023-11-28T11:08:39.546977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556901+0000","last_change":"2023-11-28T11:08:35.544292+0000","last_active":"2023-11-28T11:08:39.556901+0000","last_peered":"2023-11-28T11:08:39.556901+0000","last_clean":"2023-11-28T11:08:39.556901+0000","last_became_active":"2023-11-28T11:08:35.543905+0000","last_became_peered":"2023-11-28T11:08:35.543905+0000","last_unstale":"2023-11-28T11:08:39.556901+0000","last_undegraded":"2023-11-28T11:08:39.556901+0000","last_fullsized":"2023-11-28T11:08:39.556901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:46.079543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542701+0000","last_change":"2023-11-28T11:08:35.548959+0000","last_active":"2023-11-28T11:08:39.542701+0000","last_peered":"2023-11-28T11:08:39.542701+0000","last_clean":"2023-11-28T11:08:39.542701+0000","last_became_active":"2023-11-28T11:08:35.548707+0000","last_became_peered":"2023-11-28T11:08:35.548707+0000","last_unstale":"2023-11-28T11:08:39.542701+0000","last_undegraded":"2023-11-28T11:08:39.542701+0000","last_fullsized":"2023-11-28T11:08:39.542701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:41.426965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542575+0000","last_change":"2023-11-28T11:08:35.534783+0000","last_active":"2023-11-28T11:08:39.542575+0000","last_peered":"2023-11-28T11:08:39.542575+0000","last_clean":"2023-11-28T11:08:39.542575+0000","last_became_active":"2023-11-28T11:08:35.534520+0000","last_became_peered":"2023-11-28T11:08:35.534520+0000","last_unstale":"2023-11-28T11:08:39.542575+0000","last_undegraded":"2023-11-28T11:08:39.542575+0000","last_fullsized":"2023-11-28T11:08:39.542575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:56.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546312+0000","last_change":"2023-11-28T11:08:35.538322+0000","last_active":"2023-11-28T11:08:39.546312+0000","last_peered":"2023-11-28T11:08:39.546312+0000","last_clean":"2023-11-28T11:08:39.546312+0000","last_became_active":"2023-11-28T11:08:35.538000+0000","last_became_peered":"2023-11-28T11:08:35.538000+0000","last_unstale":"2023-11-28T11:08:39.546312+0000","last_undegraded":"2023-11-28T11:08:39.546312+0000","last_fullsized":"2023-11-28T11:08:39.546312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549399+0000","last_change":"2023-11-28T11:08:35.544771+0000","last_active":"2023-11-28T11:08:39.549399+0000","last_peered":"2023-11-28T11:08:39.549399+0000","last_clean":"2023-11-28T11:08:39.549399+0000","last_became_active":"2023-11-28T11:08:35.544525+0000","last_became_peered":"2023-11-28T11:08:35.544525+0000","last_unstale":"2023-11-28T11:08:39.549399+0000","last_undegraded":"2023-11-28T11:08:39.549399+0000","last_fullsized":"2023-11-28T11:08:39.549399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552566+0000","last_change":"2023-11-28T11:08:35.547686+0000","last_active":"2023-11-28T11:08:39.552566+0000","last_peered":"2023-11-28T11:08:39.552566+0000","last_clean":"2023-11-28T11:08:39.552566+0000","last_became_active":"2023-11-28T11:08:35.547411+0000","last_became_peered":"2023-11-28T11:08:35.547411+0000","last_unstale":"2023-11-28T11:08:39.552566+0000","last_undegraded":"2023-11-28T11:08:39.552566+0000","last_fullsized":"2023-11-28T11:08:39.552566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549830+0000","last_change":"2023-11-28T11:08:35.551541+0000","last_active":"2023-11-28T11:08:39.549830+0000","last_peered":"2023-11-28T11:08:39.549830+0000","last_clean":"2023-11-28T11:08:39.549830+0000","last_became_active":"2023-11-28T11:08:35.546691+0000","last_became_peered":"2023-11-28T11:08:35.546691+0000","last_unstale":"2023-11-28T11:08:39.549830+0000","last_undegraded":"2023-11-28T11:08:39.549830+0000","last_fullsized":"2023-11-28T11:08:39.549830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556352+0000","last_change":"2023-11-28T11:08:35.541127+0000","last_active":"2023-11-28T11:08:39.556352+0000","last_peered":"2023-11-28T11:08:39.556352+0000","last_clean":"2023-11-28T11:08:39.556352+0000","last_became_active":"2023-11-28T11:08:35.540800+0000","last_became_peered":"2023-11-28T11:08:35.540800+0000","last_unstale":"2023-11-28T11:08:39.556352+0000","last_undegraded":"2023-11-28T11:08:39.556352+0000","last_fullsized":"2023-11-28T11:08:39.556352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.543580+0000","last_change":"2023-11-28T11:08:35.546695+0000","last_active":"2023-11-28T11:08:39.543580+0000","last_peered":"2023-11-28T11:08:39.543580+0000","last_clean":"2023-11-28T11:08:39.543580+0000","last_became_active":"2023-11-28T11:08:35.546438+0000","last_became_peered":"2023-11-28T11:08:35.546438+0000","last_unstale":"2023-11-28T11:08:39.543580+0000","last_undegraded":"2023-11-28T11:08:39.543580+0000","last_fullsized":"2023-11-28T11:08:39.543580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:37.007416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556699+0000","last_change":"2023-11-28T11:08:35.548019+0000","last_active":"2023-11-28T11:08:39.556699+0000","last_peered":"2023-11-28T11:08:39.556699+0000","last_clean":"2023-11-28T11:08:39.556699+0000","last_became_active":"2023-11-28T11:08:35.541157+0000","last_became_peered":"2023-11-28T11:08:35.541157+0000","last_unstale":"2023-11-28T11:08:39.556699+0000","last_undegraded":"2023-11-28T11:08:39.556699+0000","last_fullsized":"2023-11-28T11:08:39.556699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:40.346506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551701+0000","last_change":"2023-11-28T11:08:35.540145+0000","last_active":"2023-11-28T11:08:39.551701+0000","last_peered":"2023-11-28T11:08:39.551701+0000","last_clean":"2023-11-28T11:08:39.551701+0000","last_became_active":"2023-11-28T11:08:35.539880+0000","last_became_peered":"2023-11-28T11:08:35.539880+0000","last_unstale":"2023-11-28T11:08:39.551701+0000","last_undegraded":"2023-11-28T11:08:39.551701+0000","last_fullsized":"2023-11-28T11:08:39.551701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549283+0000","last_change":"2023-11-28T11:08:35.546609+0000","last_active":"2023-11-28T11:08:39.549283+0000","last_peered":"2023-11-28T11:08:39.549283+0000","last_clean":"2023-11-28T11:08:39.549283+0000","last_became_active":"2023-11-28T11:08:35.545637+0000","last_became_peered":"2023-11-28T11:08:35.545637+0000","last_unstale":"2023-11-28T11:08:39.549283+0000","last_undegraded":"2023-11-28T11:08:39.549283+0000","last_fullsized":"2023-11-28T11:08:39.549283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544746+0000","last_change":"2023-11-28T11:08:35.540494+0000","last_active":"2023-11-28T11:08:39.544746+0000","last_peered":"2023-11-28T11:08:39.544746+0000","last_clean":"2023-11-28T11:08:39.544746+0000","last_became_active":"2023-11-28T11:08:35.539936+0000","last_became_peered":"2023-11-28T11:08:35.539936+0000","last_unstale":"2023-11-28T11:08:39.544746+0000","last_undegraded":"2023-11-28T11:08:39.544746+0000","last_fullsized":"2023-11-28T11:08:39.544746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550772+0000","last_change":"2023-11-28T11:08:35.545757+0000","last_active":"2023-11-28T11:08:39.550772+0000","last_peered":"2023-11-28T11:08:39.550772+0000","last_clean":"2023-11-28T11:08:39.550772+0000","last_became_active":"2023-11-28T11:08:35.545459+0000","last_became_peered":"2023-11-28T11:08:35.545459+0000","last_unstale":"2023-11-28T11:08:39.550772+0000","last_undegraded":"2023-11-28T11:08:39.550772+0000","last_fullsized":"2023-11-28T11:08:39.550772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556095+0000","last_change":"2023-11-28T11:08:35.540552+0000","last_active":"2023-11-28T11:08:39.556095+0000","last_peered":"2023-11-28T11:08:39.556095+0000","last_clean":"2023-11-28T11:08:39.556095+0000","last_became_active":"2023-11-28T11:08:35.539972+0000","last_became_peered":"2023-11-28T11:08:35.539972+0000","last_unstale":"2023-11-28T11:08:39.556095+0000","last_undegraded":"2023-11-28T11:08:39.556095+0000","last_fullsized":"2023-11-28T11:08:39.556095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556230+0000","last_change":"2023-11-28T11:08:35.536499+0000","last_active":"2023-11-28T11:08:39.556230+0000","last_peered":"2023-11-28T11:08:39.556230+0000","last_clean":"2023-11-28T11:08:39.556230+0000","last_became_active":"2023-11-28T11:08:35.536308+0000","last_became_peered":"2023-11-28T11:08:35.536308+0000","last_unstale":"2023-11-28T11:08:39.556230+0000","last_undegraded":"2023-11-28T11:08:39.556230+0000","last_fullsized":"2023-11-28T11:08:39.556230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546678+0000","last_change":"2023-11-28T11:08:35.542161+0000","last_active":"2023-11-28T11:08:39.546678+0000","last_peered":"2023-11-28T11:08:39.546678+0000","last_clean":"2023-11-28T11:08:39.546678+0000","last_became_active":"2023-11-28T11:08:35.541331+0000","last_became_peered":"2023-11-28T11:08:35.541331+0000","last_unstale":"2023-11-28T11:08:39.546678+0000","last_undegraded":"2023-11-28T11:08:39.546678+0000","last_fullsized":"2023-11-28T11:08:39.546678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570777+0000","last_change":"2023-11-28T11:08:39.566823+0000","last_active":"2023-11-28T11:08:39.570777+0000","last_peered":"2023-11-28T11:08:39.570777+0000","last_clean":"2023-11-28T11:08:39.570777+0000","last_became_active":"2023-11-28T11:08:39.563462+0000","last_became_peered":"2023-11-28T11:08:39.563462+0000","last_unstale":"2023-11-28T11:08:39.570777+0000","last_undegraded":"2023-11-28T11:08:39.570777+0000","last_fullsized":"2023-11-28T11:08:39.570777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:25.821485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556318+0000","last_change":"2023-11-28T11:08:35.144920+0000","last_active":"2023-11-28T11:08:39.556318+0000","last_peered":"2023-11-28T11:08:39.556318+0000","last_clean":"2023-11-28T11:08:39.556318+0000","last_became_active":"2023-11-28T11:08:35.144360+0000","last_became_peered":"2023-11-28T11:08:35.144360+0000","last_unstale":"2023-11-28T11:08:39.556318+0000","last_undegraded":"2023-11-28T11:08:39.556318+0000","last_fullsized":"2023-11-28T11:08:39.556318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544509+0000","last_change":"2023-11-28T11:08:35.537580+0000","last_active":"2023-11-28T11:08:39.544509+0000","last_peered":"2023-11-28T11:08:39.544509+0000","last_clean":"2023-11-28T11:08:39.544509+0000","last_became_active":"2023-11-28T11:08:35.537300+0000","last_became_peered":"2023-11-28T11:08:35.537300+0000","last_unstale":"2023-11-28T11:08:39.544509+0000","last_undegraded":"2023-11-28T11:08:39.544509+0000","last_fullsized":"2023-11-28T11:08:39.544509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573888+0000","last_change":"2023-11-28T11:08:39.565761+0000","last_active":"2023-11-28T11:08:39.573888+0000","last_peered":"2023-11-28T11:08:39.573888+0000","last_clean":"2023-11-28T11:08:39.573888+0000","last_became_active":"2023-11-28T11:08:39.565387+0000","last_became_peered":"2023-11-28T11:08:39.565387+0000","last_unstale":"2023-11-28T11:08:39.573888+0000","last_undegraded":"2023-11-28T11:08:39.573888+0000","last_fullsized":"2023-11-28T11:08:39.573888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:29.648074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552645+0000","last_change":"2023-11-28T11:08:35.141536+0000","last_active":"2023-11-28T11:08:39.552645+0000","last_peered":"2023-11-28T11:08:39.552645+0000","last_clean":"2023-11-28T11:08:39.552645+0000","last_became_active":"2023-11-28T11:08:35.141259+0000","last_became_peered":"2023-11-28T11:08:35.141259+0000","last_unstale":"2023-11-28T11:08:39.552645+0000","last_undegraded":"2023-11-28T11:08:39.552645+0000","last_fullsized":"2023-11-28T11:08:39.552645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551925+0000","last_change":"2023-11-28T11:08:35.551235+0000","last_active":"2023-11-28T11:08:39.551925+0000","last_peered":"2023-11-28T11:08:39.551925+0000","last_clean":"2023-11-28T11:08:39.551925+0000","last_became_active":"2023-11-28T11:08:35.550945+0000","last_became_peered":"2023-11-28T11:08:35.550945+0000","last_unstale":"2023-11-28T11:08:39.551925+0000","last_undegraded":"2023-11-28T11:08:39.551925+0000","last_fullsized":"2023-11-28T11:08:39.551925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.576247+0000","last_change":"2023-11-28T11:08:39.574297+0000","last_active":"2023-11-28T11:08:39.576247+0000","last_peered":"2023-11-28T11:08:39.576247+0000","last_clean":"2023-11-28T11:08:39.576247+0000","last_became_active":"2023-11-28T11:08:39.573836+0000","last_became_peered":"2023-11-28T11:08:39.573836+0000","last_unstale":"2023-11-28T11:08:39.576247+0000","last_undegraded":"2023-11-28T11:08:39.576247+0000","last_fullsized":"2023-11-28T11:08:39.576247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546095+0000","last_change":"2023-11-28T11:08:35.140807+0000","last_active":"2023-11-28T11:08:39.546095+0000","last_peered":"2023-11-28T11:08:39.546095+0000","last_clean":"2023-11-28T11:08:39.546095+0000","last_became_active":"2023-11-28T11:08:35.140551+0000","last_became_peered":"2023-11-28T11:08:35.140551+0000","last_unstale":"2023-11-28T11:08:39.546095+0000","last_undegraded":"2023-11-28T11:08:39.546095+0000","last_fullsized":"2023-11-28T11:08:39.546095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553480+0000","last_change":"2023-11-28T11:08:35.540497+0000","last_active":"2023-11-28T11:08:39.553480+0000","last_peered":"2023-11-28T11:08:39.553480+0000","last_clean":"2023-11-28T11:08:39.553480+0000","last_became_active":"2023-11-28T11:08:35.540154+0000","last_became_peered":"2023-11-28T11:08:35.540154+0000","last_unstale":"2023-11-28T11:08:39.553480+0000","last_undegraded":"2023-11-28T11:08:39.553480+0000","last_fullsized":"2023-11-28T11:08:39.553480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:43.354051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.574001+0000","last_change":"2023-11-28T11:08:39.568636+0000","last_active":"2023-11-28T11:08:39.574001+0000","last_peered":"2023-11-28T11:08:39.574001+0000","last_clean":"2023-11-28T11:08:39.574001+0000","last_became_active":"2023-11-28T11:08:39.568255+0000","last_became_peered":"2023-11-28T11:08:39.568255+0000","last_unstale":"2023-11-28T11:08:39.574001+0000","last_undegraded":"2023-11-28T11:08:39.574001+0000","last_fullsized":"2023-11-28T11:08:39.574001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549903+0000","last_change":"2023-11-28T11:08:35.139702+0000","last_active":"2023-11-28T11:08:39.549903+0000","last_peered":"2023-11-28T11:08:39.549903+0000","last_clean":"2023-11-28T11:08:39.549903+0000","last_became_active":"2023-11-28T11:08:35.139379+0000","last_became_peered":"2023-11-28T11:08:35.139379+0000","last_unstale":"2023-11-28T11:08:39.549903+0000","last_undegraded":"2023-11-28T11:08:39.549903+0000","last_fullsized":"2023-11-28T11:08:39.549903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546007+0000","last_change":"2023-11-28T11:08:35.532733+0000","last_active":"2023-11-28T11:08:39.546007+0000","last_peered":"2023-11-28T11:08:39.546007+0000","last_clean":"2023-11-28T11:08:39.546007+0000","last_became_active":"2023-11-28T11:08:35.532390+0000","last_became_peered":"2023-11-28T11:08:35.532390+0000","last_unstale":"2023-11-28T11:08:39.546007+0000","last_undegraded":"2023-11-28T11:08:39.546007+0000","last_fullsized":"2023-11-28T11:08:39.546007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573440+0000","last_change":"2023-11-28T11:08:39.568834+0000","last_active":"2023-11-28T11:08:39.573440+0000","last_peered":"2023-11-28T11:08:39.573440+0000","last_clean":"2023-11-28T11:08:39.573440+0000","last_became_active":"2023-11-28T11:08:39.568153+0000","last_became_peered":"2023-11-28T11:08:39.568153+0000","last_unstale":"2023-11-28T11:08:39.573440+0000","last_undegraded":"2023-11-28T11:08:39.573440+0000","last_fullsized":"2023-11-28T11:08:39.573440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546860+0000","last_change":"2023-11-28T11:08:35.140772+0000","last_active":"2023-11-28T11:08:39.546860+0000","last_peered":"2023-11-28T11:08:39.546860+0000","last_clean":"2023-11-28T11:08:39.546860+0000","last_became_active":"2023-11-28T11:08:35.140418+0000","last_became_peered":"2023-11-28T11:08:35.140418+0000","last_unstale":"2023-11-28T11:08:39.546860+0000","last_undegraded":"2023-11-28T11:08:39.546860+0000","last_fullsized":"2023-11-28T11:08:39.546860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544363+0000","last_change":"2023-11-28T11:08:35.544651+0000","last_active":"2023-11-28T11:08:39.544363+0000","last_peered":"2023-11-28T11:08:39.544363+0000","last_clean":"2023-11-28T11:08:39.544363+0000","last_became_active":"2023-11-28T11:08:35.541280+0000","last_became_peered":"2023-11-28T11:08:35.541280+0000","last_unstale":"2023-11-28T11:08:39.544363+0000","last_undegraded":"2023-11-28T11:08:39.544363+0000","last_fullsized":"2023-11-28T11:08:39.544363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568222+0000","last_change":"2023-11-28T11:08:39.562858+0000","last_active":"2023-11-28T11:08:39.568222+0000","last_peered":"2023-11-28T11:08:39.568222+0000","last_clean":"2023-11-28T11:08:39.568222+0000","last_became_active":"2023-11-28T11:08:39.562273+0000","last_became_peered":"2023-11-28T11:08:39.562273+0000","last_unstale":"2023-11-28T11:08:39.568222+0000","last_undegraded":"2023-11-28T11:08:39.568222+0000","last_fullsized":"2023-11-28T11:08:39.568222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:39.831045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552124+0000","last_change":"2023-11-28T11:08:35.144557+0000","last_active":"2023-11-28T11:08:39.552124+0000","last_peered":"2023-11-28T11:08:39.552124+0000","last_clean":"2023-11-28T11:08:39.552124+0000","last_became_active":"2023-11-28T11:08:35.144222+0000","last_became_peered":"2023-11-28T11:08:35.144222+0000","last_unstale":"2023-11-28T11:08:39.552124+0000","last_undegraded":"2023-11-28T11:08:39.552124+0000","last_fullsized":"2023-11-28T11:08:39.552124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552994+0000","last_change":"2023-11-28T11:08:35.543394+0000","last_active":"2023-11-28T11:08:39.552994+0000","last_peered":"2023-11-28T11:08:39.552994+0000","last_clean":"2023-11-28T11:08:39.552994+0000","last_became_active":"2023-11-28T11:08:35.543099+0000","last_became_peered":"2023-11-28T11:08:35.543099+0000","last_unstale":"2023-11-28T11:08:39.552994+0000","last_undegraded":"2023-11-28T11:08:39.552994+0000","last_fullsized":"2023-11-28T11:08:39.552994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571358+0000","last_change":"2023-11-28T11:08:39.570301+0000","last_active":"2023-11-28T11:08:39.571358+0000","last_peered":"2023-11-28T11:08:39.571358+0000","last_clean":"2023-11-28T11:08:39.571358+0000","last_became_active":"2023-11-28T11:08:39.569828+0000","last_became_peered":"2023-11-28T11:08:39.569828+0000","last_unstale":"2023-11-28T11:08:39.571358+0000","last_undegraded":"2023-11-28T11:08:39.571358+0000","last_fullsized":"2023-11-28T11:08:39.571358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:44.461165+0000","last_change":"2023-11-28T11:08:35.147185+0000","last_active":"2023-11-28T11:08:44.461165+0000","last_peered":"2023-11-28T11:08:44.461165+0000","last_clean":"2023-11-28T11:08:44.461165+0000","last_became_active":"2023-11-28T11:08:35.146285+0000","last_became_peered":"2023-11-28T11:08:35.146285+0000","last_unstale":"2023-11-28T11:08:44.461165+0000","last_undegraded":"2023-11-28T11:08:44.461165+0000","last_fullsized":"2023-11-28T11:08:44.461165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556565+0000","last_change":"2023-11-28T11:08:35.546895+0000","last_active":"2023-11-28T11:08:39.556565+0000","last_peered":"2023-11-28T11:08:39.556565+0000","last_clean":"2023-11-28T11:08:39.556565+0000","last_became_active":"2023-11-28T11:08:35.538447+0000","last_became_peered":"2023-11-28T11:08:35.538447+0000","last_unstale":"2023-11-28T11:08:39.556565+0000","last_undegraded":"2023-11-28T11:08:39.556565+0000","last_fullsized":"2023-11-28T11:08:39.556565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:28.436759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564343+0000","last_change":"2023-11-28T11:08:39.560554+0000","last_active":"2023-11-28T11:08:39.564343+0000","last_peered":"2023-11-28T11:08:39.564343+0000","last_clean":"2023-11-28T11:08:39.564343+0000","last_became_active":"2023-11-28T11:08:39.560032+0000","last_became_peered":"2023-11-28T11:08:39.560032+0000","last_unstale":"2023-11-28T11:08:39.564343+0000","last_undegraded":"2023-11-28T11:08:39.564343+0000","last_fullsized":"2023-11-28T11:08:39.564343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551526+0000","last_change":"2023-11-28T11:08:35.146846+0000","last_active":"2023-11-28T11:08:39.551526+0000","last_peered":"2023-11-28T11:08:39.551526+0000","last_clean":"2023-11-28T11:08:39.551526+0000","last_became_active":"2023-11-28T11:08:35.146565+0000","last_became_peered":"2023-11-28T11:08:35.146565+0000","last_unstale":"2023-11-28T11:08:39.551526+0000","last_undegraded":"2023-11-28T11:08:39.551526+0000","last_fullsized":"2023-11-28T11:08:39.551526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545959+0000","last_change":"2023-11-28T11:08:35.537655+0000","last_active":"2023-11-28T11:08:39.545959+0000","last_peered":"2023-11-28T11:08:39.545959+0000","last_clean":"2023-11-28T11:08:39.545959+0000","last_became_active":"2023-11-28T11:08:35.537401+0000","last_became_peered":"2023-11-28T11:08:35.537401+0000","last_unstale":"2023-11-28T11:08:39.545959+0000","last_undegraded":"2023-11-28T11:08:39.545959+0000","last_fullsized":"2023-11-28T11:08:39.545959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.559490+0000","last_change":"2023-11-28T11:08:39.558756+0000","last_active":"2023-11-28T11:08:39.559490+0000","last_peered":"2023-11-28T11:08:39.559490+0000","last_clean":"2023-11-28T11:08:39.559490+0000","last_became_active":"2023-11-28T11:08:39.558382+0000","last_became_peered":"2023-11-28T11:08:39.558382+0000","last_unstale":"2023-11-28T11:08:39.559490+0000","last_undegraded":"2023-11-28T11:08:39.559490+0000","last_fullsized":"2023-11-28T11:08:39.559490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:08.685769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549379+0000","last_change":"2023-11-28T11:08:35.143155+0000","last_active":"2023-11-28T11:08:39.549379+0000","last_peered":"2023-11-28T11:08:39.549379+0000","last_clean":"2023-11-28T11:08:39.549379+0000","last_became_active":"2023-11-28T11:08:35.142856+0000","last_became_peered":"2023-11-28T11:08:35.142856+0000","last_unstale":"2023-11-28T11:08:39.549379+0000","last_undegraded":"2023-11-28T11:08:39.549379+0000","last_fullsized":"2023-11-28T11:08:39.549379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546595+0000","last_change":"2023-11-28T11:08:35.550010+0000","last_active":"2023-11-28T11:08:39.546595+0000","last_peered":"2023-11-28T11:08:39.546595+0000","last_clean":"2023-11-28T11:08:39.546595+0000","last_became_active":"2023-11-28T11:08:35.548764+0000","last_became_peered":"2023-11-28T11:08:35.548764+0000","last_unstale":"2023-11-28T11:08:39.546595+0000","last_undegraded":"2023-11-28T11:08:39.546595+0000","last_fullsized":"2023-11-28T11:08:39.546595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568175+0000","last_change":"2023-11-28T11:08:39.562607+0000","last_active":"2023-11-28T11:08:39.568175+0000","last_peered":"2023-11-28T11:08:39.568175+0000","last_clean":"2023-11-28T11:08:39.568175+0000","last_became_active":"2023-11-28T11:08:39.562144+0000","last_became_peered":"2023-11-28T11:08:39.562144+0000","last_unstale":"2023-11-28T11:08:39.568175+0000","last_undegraded":"2023-11-28T11:08:39.568175+0000","last_fullsized":"2023-11-28T11:08:39.568175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:43:13.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549488+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.549488+0000","last_peered":"2023-11-28T11:08:39.549488+0000","last_clean":"2023-11-28T11:08:39.549488+0000","last_became_active":"2023-11-28T11:08:35.146468+0000","last_became_peered":"2023-11-28T11:08:35.146468+0000","last_unstale":"2023-11-28T11:08:39.549488+0000","last_undegraded":"2023-11-28T11:08:39.549488+0000","last_fullsized":"2023-11-28T11:08:39.549488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:42.348852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552146+0000","last_change":"2023-11-28T11:08:35.545831+0000","last_active":"2023-11-28T11:08:39.552146+0000","last_peered":"2023-11-28T11:08:39.552146+0000","last_clean":"2023-11-28T11:08:39.552146+0000","last_became_active":"2023-11-28T11:08:35.544509+0000","last_became_peered":"2023-11-28T11:08:35.544509+0000","last_unstale":"2023-11-28T11:08:39.552146+0000","last_undegraded":"2023-11-28T11:08:39.552146+0000","last_fullsized":"2023-11-28T11:08:39.552146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564184+0000","last_change":"2023-11-28T11:08:39.560957+0000","last_active":"2023-11-28T11:08:39.564184+0000","last_peered":"2023-11-28T11:08:39.564184+0000","last_clean":"2023-11-28T11:08:39.564184+0000","last_became_active":"2023-11-28T11:08:39.560011+0000","last_became_peered":"2023-11-28T11:08:39.560011+0000","last_unstale":"2023-11-28T11:08:39.564184+0000","last_undegraded":"2023-11-28T11:08:39.564184+0000","last_fullsized":"2023-11-28T11:08:39.564184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551361+0000","last_change":"2023-11-28T11:08:35.137486+0000","last_active":"2023-11-28T11:08:39.551361+0000","last_peered":"2023-11-28T11:08:39.551361+0000","last_clean":"2023-11-28T11:08:39.551361+0000","last_became_active":"2023-11-28T11:08:35.137192+0000","last_became_peered":"2023-11-28T11:08:35.137192+0000","last_unstale":"2023-11-28T11:08:39.551361+0000","last_undegraded":"2023-11-28T11:08:39.551361+0000","last_fullsized":"2023-11-28T11:08:39.551361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:09.971307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545264+0000","last_change":"2023-11-28T11:08:35.543758+0000","last_active":"2023-11-28T11:08:39.545264+0000","last_peered":"2023-11-28T11:08:39.545264+0000","last_clean":"2023-11-28T11:08:39.545264+0000","last_became_active":"2023-11-28T11:08:35.543535+0000","last_became_peered":"2023-11-28T11:08:35.543535+0000","last_unstale":"2023-11-28T11:08:39.545264+0000","last_undegraded":"2023-11-28T11:08:39.545264+0000","last_fullsized":"2023-11-28T11:08:39.545264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570639+0000","last_change":"2023-11-28T11:08:39.567742+0000","last_active":"2023-11-28T11:08:39.570639+0000","last_peered":"2023-11-28T11:08:39.570639+0000","last_clean":"2023-11-28T11:08:39.570639+0000","last_became_active":"2023-11-28T11:08:39.567313+0000","last_became_peered":"2023-11-28T11:08:39.567313+0000","last_unstale":"2023-11-28T11:08:39.570639+0000","last_undegraded":"2023-11-28T11:08:39.570639+0000","last_fullsized":"2023-11-28T11:08:39.570639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552255+0000","last_change":"2023-11-28T11:08:35.142255+0000","last_active":"2023-11-28T11:08:39.552255+0000","last_peered":"2023-11-28T11:08:39.552255+0000","last_clean":"2023-11-28T11:08:39.552255+0000","last_became_active":"2023-11-28T11:08:35.140243+0000","last_became_peered":"2023-11-28T11:08:35.140243+0000","last_unstale":"2023-11-28T11:08:39.552255+0000","last_undegraded":"2023-11-28T11:08:39.552255+0000","last_fullsized":"2023-11-28T11:08:39.552255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:43.134431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546061+0000","last_change":"2023-11-28T11:08:35.540595+0000","last_active":"2023-11-28T11:08:39.546061+0000","last_peered":"2023-11-28T11:08:39.546061+0000","last_clean":"2023-11-28T11:08:39.546061+0000","last_became_active":"2023-11-28T11:08:35.540303+0000","last_became_peered":"2023-11-28T11:08:35.540303+0000","last_unstale":"2023-11-28T11:08:39.546061+0000","last_undegraded":"2023-11-28T11:08:39.546061+0000","last_fullsized":"2023-11-28T11:08:39.546061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562667+0000","last_change":"2023-11-28T11:08:39.560601+0000","last_active":"2023-11-28T11:08:39.562667+0000","last_peered":"2023-11-28T11:08:39.562667+0000","last_clean":"2023-11-28T11:08:39.562667+0000","last_became_active":"2023-11-28T11:08:39.560147+0000","last_became_peered":"2023-11-28T11:08:39.560147+0000","last_unstale":"2023-11-28T11:08:39.562667+0000","last_undegraded":"2023-11-28T11:08:39.562667+0000","last_fullsized":"2023-11-28T11:08:39.562667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:39.607355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552185+0000","last_change":"2023-11-28T11:08:35.142804+0000","last_active":"2023-11-28T11:08:39.552185+0000","last_peered":"2023-11-28T11:08:39.552185+0000","last_clean":"2023-11-28T11:08:39.552185+0000","last_became_active":"2023-11-28T11:08:35.142407+0000","last_became_peered":"2023-11-28T11:08:35.142407+0000","last_unstale":"2023-11-28T11:08:39.552185+0000","last_undegraded":"2023-11-28T11:08:39.552185+0000","last_fullsized":"2023-11-28T11:08:39.552185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:12.791046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560104+0000","last_change":"2023-11-28T11:08:35.543448+0000","last_active":"2023-11-28T11:08:39.560104+0000","last_peered":"2023-11-28T11:08:39.560104+0000","last_clean":"2023-11-28T11:08:39.560104+0000","last_became_active":"2023-11-28T11:08:35.541094+0000","last_became_peered":"2023-11-28T11:08:35.541094+0000","last_unstale":"2023-11-28T11:08:39.560104+0000","last_undegraded":"2023-11-28T11:08:39.560104+0000","last_fullsized":"2023-11-28T11:08:39.560104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571651+0000","last_change":"2023-11-28T11:08:39.562935+0000","last_active":"2023-11-28T11:08:39.571651+0000","last_peered":"2023-11-28T11:08:39.571651+0000","last_clean":"2023-11-28T11:08:39.571651+0000","last_became_active":"2023-11-28T11:08:39.562529+0000","last_became_peered":"2023-11-28T11:08:39.562529+0000","last_unstale":"2023-11-28T11:08:39.571651+0000","last_undegraded":"2023-11-28T11:08:39.571651+0000","last_fullsized":"2023-11-28T11:08:39.571651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:31.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552244+0000","last_change":"2023-11-28T11:08:35.143035+0000","last_active":"2023-11-28T11:08:39.552244+0000","last_peered":"2023-11-28T11:08:39.552244+0000","last_clean":"2023-11-28T11:08:39.552244+0000","last_became_active":"2023-11-28T11:08:35.142697+0000","last_became_peered":"2023-11-28T11:08:35.142697+0000","last_unstale":"2023-11-28T11:08:39.552244+0000","last_undegraded":"2023-11-28T11:08:39.552244+0000","last_fullsized":"2023-11-28T11:08:39.552244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:24.895027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552639+0000","last_change":"2023-11-28T11:08:35.535083+0000","last_active":"2023-11-28T11:08:39.552639+0000","last_peered":"2023-11-28T11:08:39.552639+0000","last_clean":"2023-11-28T11:08:39.552639+0000","last_became_active":"2023-11-28T11:08:35.534800+0000","last_became_peered":"2023-11-28T11:08:35.534800+0000","last_unstale":"2023-11-28T11:08:39.552639+0000","last_undegraded":"2023-11-28T11:08:39.552639+0000","last_fullsized":"2023-11-28T11:08:39.552639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571669+0000","last_change":"2023-11-28T11:08:39.565208+0000","last_active":"2023-11-28T11:08:39.571669+0000","last_peered":"2023-11-28T11:08:39.571669+0000","last_clean":"2023-11-28T11:08:39.571669+0000","last_became_active":"2023-11-28T11:08:39.564299+0000","last_became_peered":"2023-11-28T11:08:39.564299+0000","last_unstale":"2023-11-28T11:08:39.571669+0000","last_undegraded":"2023-11-28T11:08:39.571669+0000","last_fullsized":"2023-11-28T11:08:39.571669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:48.486752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546343+0000","last_change":"2023-11-28T11:08:35.141725+0000","last_active":"2023-11-28T11:08:39.546343+0000","last_peered":"2023-11-28T11:08:39.546343+0000","last_clean":"2023-11-28T11:08:39.546343+0000","last_became_active":"2023-11-28T11:08:35.141445+0000","last_became_peered":"2023-11-28T11:08:35.141445+0000","last_unstale":"2023-11-28T11:08:39.546343+0000","last_undegraded":"2023-11-28T11:08:39.546343+0000","last_fullsized":"2023-11-28T11:08:39.546343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556057+0000","last_change":"2023-11-28T11:08:35.548819+0000","last_active":"2023-11-28T11:08:39.556057+0000","last_peered":"2023-11-28T11:08:39.556057+0000","last_clean":"2023-11-28T11:08:39.556057+0000","last_became_active":"2023-11-28T11:08:35.548403+0000","last_became_peered":"2023-11-28T11:08:35.548403+0000","last_unstale":"2023-11-28T11:08:39.556057+0000","last_undegraded":"2023-11-28T11:08:39.556057+0000","last_fullsized":"2023-11-28T11:08:39.556057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571605+0000","last_change":"2023-11-28T11:08:39.562941+0000","last_active":"2023-11-28T11:08:39.571605+0000","last_peered":"2023-11-28T11:08:39.571605+0000","last_clean":"2023-11-28T11:08:39.571605+0000","last_became_active":"2023-11-28T11:08:39.562555+0000","last_became_peered":"2023-11-28T11:08:39.562555+0000","last_unstale":"2023-11-28T11:08:39.571605+0000","last_undegraded":"2023-11-28T11:08:39.571605+0000","last_fullsized":"2023-11-28T11:08:39.571605+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544942+0000","last_change":"2023-11-28T11:08:35.142283+0000","last_active":"2023-11-28T11:08:39.544942+0000","last_peered":"2023-11-28T11:08:39.544942+0000","last_clean":"2023-11-28T11:08:39.544942+0000","last_became_active":"2023-11-28T11:08:35.141976+0000","last_became_peered":"2023-11-28T11:08:35.141976+0000","last_unstale":"2023-11-28T11:08:39.544942+0000","last_undegraded":"2023-11-28T11:08:39.544942+0000","last_fullsized":"2023-11-28T11:08:39.544942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556613+0000","last_change":"2023-11-28T11:08:35.540882+0000","last_active":"2023-11-28T11:08:39.556613+0000","last_peered":"2023-11-28T11:08:39.556613+0000","last_clean":"2023-11-28T11:08:39.556613+0000","last_became_active":"2023-11-28T11:08:35.540566+0000","last_became_peered":"2023-11-28T11:08:35.540566+0000","last_unstale":"2023-11-28T11:08:39.556613+0000","last_undegraded":"2023-11-28T11:08:39.556613+0000","last_fullsized":"2023-11-28T11:08:39.556613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570782+0000","last_change":"2023-11-28T11:08:39.566349+0000","last_active":"2023-11-28T11:08:39.570782+0000","last_peered":"2023-11-28T11:08:39.570782+0000","last_clean":"2023-11-28T11:08:39.570782+0000","last_became_active":"2023-11-28T11:08:39.565923+0000","last_became_peered":"2023-11-28T11:08:39.565923+0000","last_unstale":"2023-11-28T11:08:39.570782+0000","last_undegraded":"2023-11-28T11:08:39.570782+0000","last_fullsized":"2023-11-28T11:08:39.570782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551243+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.551243+0000","last_peered":"2023-11-28T11:08:39.551243+0000","last_clean":"2023-11-28T11:08:39.551243+0000","last_became_active":"2023-11-28T11:08:35.146466+0000","last_became_peered":"2023-11-28T11:08:35.146466+0000","last_unstale":"2023-11-28T11:08:39.551243+0000","last_undegraded":"2023-11-28T11:08:39.551243+0000","last_fullsized":"2023-11-28T11:08:39.551243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556761+0000","last_change":"2023-11-28T11:08:35.540325+0000","last_active":"2023-11-28T11:08:39.556761+0000","last_peered":"2023-11-28T11:08:39.556761+0000","last_clean":"2023-11-28T11:08:39.556761+0000","last_became_active":"2023-11-28T11:08:35.540045+0000","last_became_peered":"2023-11-28T11:08:35.540045+0000","last_unstale":"2023-11-28T11:08:39.556761+0000","last_undegraded":"2023-11-28T11:08:39.556761+0000","last_fullsized":"2023-11-28T11:08:39.556761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566338+0000","last_change":"2023-11-28T11:08:39.562874+0000","last_active":"2023-11-28T11:08:39.566338+0000","last_peered":"2023-11-28T11:08:39.566338+0000","last_clean":"2023-11-28T11:08:39.566338+0000","last_became_active":"2023-11-28T11:08:39.562484+0000","last_became_peered":"2023-11-28T11:08:39.562484+0000","last_unstale":"2023-11-28T11:08:39.566338+0000","last_undegraded":"2023-11-28T11:08:39.566338+0000","last_fullsized":"2023-11-28T11:08:39.566338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552660+0000","last_change":"2023-11-28T11:08:35.139637+0000","last_active":"2023-11-28T11:08:39.552660+0000","last_peered":"2023-11-28T11:08:39.552660+0000","last_clean":"2023-11-28T11:08:39.552660+0000","last_became_active":"2023-11-28T11:08:35.138268+0000","last_became_peered":"2023-11-28T11:08:35.138268+0000","last_unstale":"2023-11-28T11:08:39.552660+0000","last_undegraded":"2023-11-28T11:08:39.552660+0000","last_fullsized":"2023-11-28T11:08:39.552660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556539+0000","last_change":"2023-11-28T11:08:35.543314+0000","last_active":"2023-11-28T11:08:39.556539+0000","last_peered":"2023-11-28T11:08:39.556539+0000","last_clean":"2023-11-28T11:08:39.556539+0000","last_became_active":"2023-11-28T11:08:35.539838+0000","last_became_peered":"2023-11-28T11:08:35.539838+0000","last_unstale":"2023-11-28T11:08:39.556539+0000","last_undegraded":"2023-11-28T11:08:39.556539+0000","last_fullsized":"2023-11-28T11:08:39.556539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:53.884226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568846+0000","last_change":"2023-11-28T11:08:39.565918+0000","last_active":"2023-11-28T11:08:39.568846+0000","last_peered":"2023-11-28T11:08:39.568846+0000","last_clean":"2023-11-28T11:08:39.568846+0000","last_became_active":"2023-11-28T11:08:39.562911+0000","last_became_peered":"2023-11-28T11:08:39.562911+0000","last_unstale":"2023-11-28T11:08:39.568846+0000","last_undegraded":"2023-11-28T11:08:39.568846+0000","last_fullsized":"2023-11-28T11:08:39.568846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:11:09.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546326+0000","last_change":"2023-11-28T11:08:35.143397+0000","last_active":"2023-11-28T11:08:39.546326+0000","last_peered":"2023-11-28T11:08:39.546326+0000","last_clean":"2023-11-28T11:08:39.546326+0000","last_became_active":"2023-11-28T11:08:35.142466+0000","last_became_peered":"2023-11-28T11:08:35.142466+0000","last_unstale":"2023-11-28T11:08:39.546326+0000","last_undegraded":"2023-11-28T11:08:39.546326+0000","last_fullsized":"2023-11-28T11:08:39.546326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556168+0000","last_change":"2023-11-28T11:08:35.534626+0000","last_active":"2023-11-28T11:08:39.556168+0000","last_peered":"2023-11-28T11:08:39.556168+0000","last_clean":"2023-11-28T11:08:39.556168+0000","last_became_active":"2023-11-28T11:08:35.534379+0000","last_became_peered":"2023-11-28T11:08:35.534379+0000","last_unstale":"2023-11-28T11:08:39.556168+0000","last_undegraded":"2023-11-28T11:08:39.556168+0000","last_fullsized":"2023-11-28T11:08:39.556168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573390+0000","last_change":"2023-11-28T11:08:39.569661+0000","last_active":"2023-11-28T11:08:39.573390+0000","last_peered":"2023-11-28T11:08:39.573390+0000","last_clean":"2023-11-28T11:08:39.573390+0000","last_became_active":"2023-11-28T11:08:39.569175+0000","last_became_peered":"2023-11-28T11:08:39.569175+0000","last_unstale":"2023-11-28T11:08:39.573390+0000","last_undegraded":"2023-11-28T11:08:39.573390+0000","last_fullsized":"2023-11-28T11:08:39.573390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:59.446120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544982+0000","last_change":"2023-11-28T11:08:35.143399+0000","last_active":"2023-11-28T11:08:39.544982+0000","last_peered":"2023-11-28T11:08:39.544982+0000","last_clean":"2023-11-28T11:08:39.544982+0000","last_became_active":"2023-11-28T11:08:35.143102+0000","last_became_peered":"2023-11-28T11:08:35.143102+0000","last_unstale":"2023-11-28T11:08:39.544982+0000","last_undegraded":"2023-11-28T11:08:39.544982+0000","last_fullsized":"2023-11-28T11:08:39.544982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556842+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:08:39.556842+0000","last_peered":"2023-11-28T11:08:39.556842+0000","last_clean":"2023-11-28T11:08:39.556842+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:08:39.556842+0000","last_undegraded":"2023-11-28T11:08:39.556842+0000","last_fullsized":"2023-11-28T11:08:39.556842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545915+0000","last_change":"2023-11-28T11:08:35.541344+0000","last_active":"2023-11-28T11:08:39.545915+0000","last_peered":"2023-11-28T11:08:39.545915+0000","last_clean":"2023-11-28T11:08:39.545915+0000","last_became_active":"2023-11-28T11:08:35.541086+0000","last_became_peered":"2023-11-28T11:08:35.541086+0000","last_unstale":"2023-11-28T11:08:39.545915+0000","last_undegraded":"2023-11-28T11:08:39.545915+0000","last_fullsized":"2023-11-28T11:08:39.545915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.572142+0000","last_change":"2023-11-28T11:08:39.567333+0000","last_active":"2023-11-28T11:08:39.572142+0000","last_peered":"2023-11-28T11:08:39.572142+0000","last_clean":"2023-11-28T11:08:39.572142+0000","last_became_active":"2023-11-28T11:08:39.566952+0000","last_became_peered":"2023-11-28T11:08:39.566952+0000","last_unstale":"2023-11-28T11:08:39.572142+0000","last_undegraded":"2023-11-28T11:08:39.572142+0000","last_fullsized":"2023-11-28T11:08:39.572142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545875+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:08:39.545875+0000","last_peered":"2023-11-28T11:08:39.545875+0000","last_clean":"2023-11-28T11:08:39.545875+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:08:39.545875+0000","last_undegraded":"2023-11-28T11:08:39.545875+0000","last_fullsized":"2023-11-28T11:08:39.545875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:40.959549+0000","last_change":"2023-11-28T11:08:35.138175+0000","last_active":"2023-11-28T11:08:40.959549+0000","last_peered":"2023-11-28T11:08:40.959549+0000","last_clean":"2023-11-28T11:08:40.959549+0000","last_became_active":"2023-11-28T11:08:35.137577+0000","last_became_peered":"2023-11-28T11:08:35.137577+0000","last_unstale":"2023-11-28T11:08:40.959549+0000","last_undegraded":"2023-11-28T11:08:40.959549+0000","last_fullsized":"2023-11-28T11:08:40.959549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560017+0000","last_change":"2023-11-28T11:08:35.540521+0000","last_active":"2023-11-28T11:08:39.560017+0000","last_peered":"2023-11-28T11:08:39.560017+0000","last_clean":"2023-11-28T11:08:39.560017+0000","last_became_active":"2023-11-28T11:08:35.538206+0000","last_became_peered":"2023-11-28T11:08:35.538206+0000","last_unstale":"2023-11-28T11:08:39.560017+0000","last_undegraded":"2023-11-28T11:08:39.560017+0000","last_fullsized":"2023-11-28T11:08:39.560017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568578+0000","last_change":"2023-11-28T11:08:39.565755+0000","last_active":"2023-11-28T11:08:39.568578+0000","last_peered":"2023-11-28T11:08:39.568578+0000","last_clean":"2023-11-28T11:08:39.568578+0000","last_became_active":"2023-11-28T11:08:39.565093+0000","last_became_peered":"2023-11-28T11:08:39.565093+0000","last_unstale":"2023-11-28T11:08:39.568578+0000","last_undegraded":"2023-11-28T11:08:39.568578+0000","last_fullsized":"2023-11-28T11:08:39.568578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555643+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:08:39.555643+0000","last_peered":"2023-11-28T11:08:39.555643+0000","last_clean":"2023-11-28T11:08:39.555643+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:08:39.555643+0000","last_undegraded":"2023-11-28T11:08:39.555643+0000","last_fullsized":"2023-11-28T11:08:39.555643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550860+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:08:39.550860+0000","last_peered":"2023-11-28T11:08:39.550860+0000","last_clean":"2023-11-28T11:08:39.550860+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:08:39.550860+0000","last_undegraded":"2023-11-28T11:08:39.550860+0000","last_fullsized":"2023-11-28T11:08:39.550860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549175+0000","last_change":"2023-11-28T11:08:35.146578+0000","last_active":"2023-11-28T11:08:39.549175+0000","last_peered":"2023-11-28T11:08:39.549175+0000","last_clean":"2023-11-28T11:08:39.549175+0000","last_became_active":"2023-11-28T11:08:35.146300+0000","last_became_peered":"2023-11-28T11:08:35.146300+0000","last_unstale":"2023-11-28T11:08:39.549175+0000","last_undegraded":"2023-11-28T11:08:39.549175+0000","last_fullsized":"2023-11-28T11:08:39.549175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552070+0000","last_change":"2023-11-28T11:08:35.545787+0000","last_active":"2023-11-28T11:08:39.552070+0000","last_peered":"2023-11-28T11:08:39.552070+0000","last_clean":"2023-11-28T11:08:39.552070+0000","last_became_active":"2023-11-28T11:08:35.544359+0000","last_became_peered":"2023-11-28T11:08:35.544359+0000","last_unstale":"2023-11-28T11:08:39.552070+0000","last_undegraded":"2023-11-28T11:08:39.552070+0000","last_fullsized":"2023-11-28T11:08:39.552070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.561183+0000","last_change":"2023-11-28T11:08:39.558516+0000","last_active":"2023-11-28T11:08:39.561183+0000","last_peered":"2023-11-28T11:08:39.561183+0000","last_clean":"2023-11-28T11:08:39.561183+0000","last_became_active":"2023-11-28T11:08:39.558117+0000","last_became_peered":"2023-11-28T11:08:39.558117+0000","last_unstale":"2023-11-28T11:08:39.561183+0000","last_undegraded":"2023-11-28T11:08:39.561183+0000","last_fullsized":"2023-11-28T11:08:39.561183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551550+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:08:39.551550+0000","last_peered":"2023-11-28T11:08:39.551550+0000","last_clean":"2023-11-28T11:08:39.551550+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:08:39.551550+0000","last_undegraded":"2023-11-28T11:08:39.551550+0000","last_fullsized":"2023-11-28T11:08:39.551550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.554459+0000","last_change":"2023-11-28T11:08:35.134990+0000","last_active":"2023-11-28T11:08:39.554459+0000","last_peered":"2023-11-28T11:08:39.554459+0000","last_clean":"2023-11-28T11:08:39.554459+0000","last_became_active":"2023-11-28T11:08:35.134728+0000","last_became_peered":"2023-11-28T11:08:35.134728+0000","last_unstale":"2023-11-28T11:08:39.554459+0000","last_undegraded":"2023-11-28T11:08:39.554459+0000","last_fullsized":"2023-11-28T11:08:39.554459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556117+0000","last_change":"2023-11-28T11:08:35.540730+0000","last_active":"2023-11-28T11:08:39.556117+0000","last_peered":"2023-11-28T11:08:39.556117+0000","last_clean":"2023-11-28T11:08:39.556117+0000","last_became_active":"2023-11-28T11:08:35.540312+0000","last_became_peered":"2023-11-28T11:08:35.540312+0000","last_unstale":"2023-11-28T11:08:39.556117+0000","last_undegraded":"2023-11-28T11:08:39.556117+0000","last_fullsized":"2023-11-28T11:08:39.556117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568532+0000","last_change":"2023-11-28T11:08:39.561818+0000","last_active":"2023-11-28T11:08:39.568532+0000","last_peered":"2023-11-28T11:08:39.568532+0000","last_clean":"2023-11-28T11:08:39.568532+0000","last_became_active":"2023-11-28T11:08:39.561292+0000","last_became_peered":"2023-11-28T11:08:39.561292+0000","last_unstale":"2023-11-28T11:08:39.568532+0000","last_undegraded":"2023-11-28T11:08:39.568532+0000","last_fullsized":"2023-11-28T11:08:39.568532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550685+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:08:39.550685+0000","last_peered":"2023-11-28T11:08:39.550685+0000","last_clean":"2023-11-28T11:08:39.550685+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:08:39.550685+0000","last_undegraded":"2023-11-28T11:08:39.550685+0000","last_fullsized":"2023-11-28T11:08:39.550685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546311+0000","last_change":"2023-11-28T11:08:35.140783+0000","last_active":"2023-11-28T11:08:39.546311+0000","last_peered":"2023-11-28T11:08:39.546311+0000","last_clean":"2023-11-28T11:08:39.546311+0000","last_became_active":"2023-11-28T11:08:35.140512+0000","last_became_peered":"2023-11-28T11:08:35.140512+0000","last_unstale":"2023-11-28T11:08:39.546311+0000","last_undegraded":"2023-11-28T11:08:39.546311+0000","last_fullsized":"2023-11-28T11:08:39.546311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551352+0000","last_change":"2023-11-28T11:08:35.547824+0000","last_active":"2023-11-28T11:08:39.551352+0000","last_peered":"2023-11-28T11:08:39.551352+0000","last_clean":"2023-11-28T11:08:39.551352+0000","last_became_active":"2023-11-28T11:08:35.547517+0000","last_became_peered":"2023-11-28T11:08:35.547517+0000","last_unstale":"2023-11-28T11:08:39.551352+0000","last_undegraded":"2023-11-28T11:08:39.551352+0000","last_fullsized":"2023-11-28T11:08:39.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:55.715375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570503+0000","last_change":"2023-11-28T11:08:39.564906+0000","last_active":"2023-11-28T11:08:39.570503+0000","last_peered":"2023-11-28T11:08:39.570503+0000","last_clean":"2023-11-28T11:08:39.570503+0000","last_became_active":"2023-11-28T11:08:39.564258+0000","last_became_peered":"2023-11-28T11:08:39.564258+0000","last_unstale":"2023-11-28T11:08:39.570503+0000","last_undegraded":"2023-11-28T11:08:39.570503+0000","last_fullsized":"2023-11-28T11:08:39.570503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546086+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:08:39.546086+0000","last_peered":"2023-11-28T11:08:39.546086+0000","last_clean":"2023-11-28T11:08:39.546086+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:08:39.546086+0000","last_undegraded":"2023-11-28T11:08:39.546086+0000","last_fullsized":"2023-11-28T11:08:39.546086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556375+0000","last_change":"2023-11-28T11:08:35.144005+0000","last_active":"2023-11-28T11:08:39.556375+0000","last_peered":"2023-11-28T11:08:39.556375+0000","last_clean":"2023-11-28T11:08:39.556375+0000","last_became_active":"2023-11-28T11:08:35.141190+0000","last_became_peered":"2023-11-28T11:08:35.141190+0000","last_unstale":"2023-11-28T11:08:39.556375+0000","last_undegraded":"2023-11-28T11:08:39.556375+0000","last_fullsized":"2023-11-28T11:08:39.556375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555324+0000","last_change":"2023-11-28T11:08:35.539604+0000","last_active":"2023-11-28T11:08:39.555324+0000","last_peered":"2023-11-28T11:08:39.555324+0000","last_clean":"2023-11-28T11:08:39.555324+0000","last_became_active":"2023-11-28T11:08:35.539319+0000","last_became_peered":"2023-11-28T11:08:35.539319+0000","last_unstale":"2023-11-28T11:08:39.555324+0000","last_undegraded":"2023-11-28T11:08:39.555324+0000","last_fullsized":"2023-11-28T11:08:39.555324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562105+0000","last_change":"2023-11-28T11:08:39.559157+0000","last_active":"2023-11-28T11:08:39.562105+0000","last_peered":"2023-11-28T11:08:39.562105+0000","last_clean":"2023-11-28T11:08:39.562105+0000","last_became_active":"2023-11-28T11:08:39.558790+0000","last_became_peered":"2023-11-28T11:08:39.558790+0000","last_unstale":"2023-11-28T11:08:39.562105+0000","last_undegraded":"2023-11-28T11:08:39.562105+0000","last_fullsized":"2023-11-28T11:08:39.562105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:18.469803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550458+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:08:39.550458+0000","last_peered":"2023-11-28T11:08:39.550458+0000","last_clean":"2023-11-28T11:08:39.550458+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:08:39.550458+0000","last_undegraded":"2023-11-28T11:08:39.550458+0000","last_fullsized":"2023-11-28T11:08:39.550458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555493+0000","last_change":"2023-11-28T11:08:35.136293+0000","last_active":"2023-11-28T11:08:39.555493+0000","last_peered":"2023-11-28T11:08:39.555493+0000","last_clean":"2023-11-28T11:08:39.555493+0000","last_became_active":"2023-11-28T11:08:35.136011+0000","last_became_peered":"2023-11-28T11:08:35.136011+0000","last_unstale":"2023-11-28T11:08:39.555493+0000","last_undegraded":"2023-11-28T11:08:39.555493+0000","last_fullsized":"2023-11-28T11:08:39.555493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556312+0000","last_change":"2023-11-28T11:08:35.548876+0000","last_active":"2023-11-28T11:08:39.556312+0000","last_peered":"2023-11-28T11:08:39.556312+0000","last_clean":"2023-11-28T11:08:39.556312+0000","last_became_active":"2023-11-28T11:08:35.548606+0000","last_became_peered":"2023-11-28T11:08:35.548606+0000","last_unstale":"2023-11-28T11:08:39.556312+0000","last_undegraded":"2023-11-28T11:08:39.556312+0000","last_fullsized":"2023-11-28T11:08:39.556312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570495+0000","last_change":"2023-11-28T11:08:39.564356+0000","last_active":"2023-11-28T11:08:39.570495+0000","last_peered":"2023-11-28T11:08:39.570495+0000","last_clean":"2023-11-28T11:08:39.570495+0000","last_became_active":"2023-11-28T11:08:39.563998+0000","last_became_peered":"2023-11-28T11:08:39.563998+0000","last_unstale":"2023-11-28T11:08:39.570495+0000","last_undegraded":"2023-11-28T11:08:39.570495+0000","last_fullsized":"2023-11-28T11:08:39.570495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553351+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:08:39.553351+0000","last_peered":"2023-11-28T11:08:39.553351+0000","last_clean":"2023-11-28T11:08:39.553351+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:08:39.553351+0000","last_undegraded":"2023-11-28T11:08:39.553351+0000","last_fullsized":"2023-11-28T11:08:39.553351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546501+0000","last_change":"2023-11-28T11:08:35.141912+0000","last_active":"2023-11-28T11:08:39.546501+0000","last_peered":"2023-11-28T11:08:39.546501+0000","last_clean":"2023-11-28T11:08:39.546501+0000","last_became_active":"2023-11-28T11:08:35.141586+0000","last_became_peered":"2023-11-28T11:08:35.141586+0000","last_unstale":"2023-11-28T11:08:39.546501+0000","last_undegraded":"2023-11-28T11:08:39.546501+0000","last_fullsized":"2023-11-28T11:08:39.546501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552908+0000","last_change":"2023-11-28T11:08:35.542267+0000","last_active":"2023-11-28T11:08:39.552908+0000","last_peered":"2023-11-28T11:08:39.552908+0000","last_clean":"2023-11-28T11:08:39.552908+0000","last_became_active":"2023-11-28T11:08:35.541323+0000","last_became_peered":"2023-11-28T11:08:35.541323+0000","last_unstale":"2023-11-28T11:08:39.552908+0000","last_undegraded":"2023-11-28T11:08:39.552908+0000","last_fullsized":"2023-11-28T11:08:39.552908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570775+0000","last_change":"2023-11-28T11:08:39.566246+0000","last_active":"2023-11-28T11:08:39.570775+0000","last_peered":"2023-11-28T11:08:39.570775+0000","last_clean":"2023-11-28T11:08:39.570775+0000","last_became_active":"2023-11-28T11:08:39.565895+0000","last_became_peered":"2023-11-28T11:08:39.565895+0000","last_unstale":"2023-11-28T11:08:39.570775+0000","last_undegraded":"2023-11-28T11:08:39.570775+0000","last_fullsized":"2023-11-28T11:08:39.570775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556229+0000","last_change":"2023-11-28T11:08:35.146576+0000","last_active":"2023-11-28T11:08:39.556229+0000","last_peered":"2023-11-28T11:08:39.556229+0000","last_clean":"2023-11-28T11:08:39.556229+0000","last_became_active":"2023-11-28T11:08:35.146283+0000","last_became_peered":"2023-11-28T11:08:35.146283+0000","last_unstale":"2023-11-28T11:08:39.556229+0000","last_undegraded":"2023-11-28T11:08:39.556229+0000","last_fullsized":"2023-11-28T11:08:39.556229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542850+0000","last_change":"2023-11-28T11:08:35.536348+0000","last_active":"2023-11-28T11:08:39.542850+0000","last_peered":"2023-11-28T11:08:39.542850+0000","last_clean":"2023-11-28T11:08:39.542850+0000","last_became_active":"2023-11-28T11:08:35.536095+0000","last_became_peered":"2023-11-28T11:08:35.536095+0000","last_unstale":"2023-11-28T11:08:39.542850+0000","last_undegraded":"2023-11-28T11:08:39.542850+0000","last_fullsized":"2023-11-28T11:08:39.542850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.991268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570654+0000","last_change":"2023-11-28T11:08:39.567740+0000","last_active":"2023-11-28T11:08:39.570654+0000","last_peered":"2023-11-28T11:08:39.570654+0000","last_clean":"2023-11-28T11:08:39.570654+0000","last_became_active":"2023-11-28T11:08:39.567279+0000","last_became_peered":"2023-11-28T11:08:39.567279+0000","last_unstale":"2023-11-28T11:08:39.570654+0000","last_undegraded":"2023-11-28T11:08:39.570654+0000","last_fullsized":"2023-11-28T11:08:39.570654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546145+0000","last_change":"2023-11-28T11:08:35.141111+0000","last_active":"2023-11-28T11:08:39.546145+0000","last_peered":"2023-11-28T11:08:39.546145+0000","last_clean":"2023-11-28T11:08:39.546145+0000","last_became_active":"2023-11-28T11:08:35.140804+0000","last_became_peered":"2023-11-28T11:08:35.140804+0000","last_unstale":"2023-11-28T11:08:39.546145+0000","last_undegraded":"2023-11-28T11:08:39.546145+0000","last_fullsized":"2023-11-28T11:08:39.546145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556281+0000","last_change":"2023-11-28T11:08:35.547330+0000","last_active":"2023-11-28T11:08:39.556281+0000","last_peered":"2023-11-28T11:08:39.556281+0000","last_clean":"2023-11-28T11:08:39.556281+0000","last_became_active":"2023-11-28T11:08:35.546970+0000","last_became_peered":"2023-11-28T11:08:35.546970+0000","last_unstale":"2023-11-28T11:08:39.556281+0000","last_undegraded":"2023-11-28T11:08:39.556281+0000","last_fullsized":"2023-11-28T11:08:39.556281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:28.259978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568054+0000","last_change":"2023-11-28T11:08:39.565657+0000","last_active":"2023-11-28T11:08:39.568054+0000","last_peered":"2023-11-28T11:08:39.568054+0000","last_clean":"2023-11-28T11:08:39.568054+0000","last_became_active":"2023-11-28T11:08:39.565004+0000","last_became_peered":"2023-11-28T11:08:39.565004+0000","last_unstale":"2023-11-28T11:08:39.568054+0000","last_undegraded":"2023-11-28T11:08:39.568054+0000","last_fullsized":"2023-11-28T11:08:39.568054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546414+0000","last_change":"2023-11-28T11:08:35.146886+0000","last_active":"2023-11-28T11:08:39.546414+0000","last_peered":"2023-11-28T11:08:39.546414+0000","last_clean":"2023-11-28T11:08:39.546414+0000","last_became_active":"2023-11-28T11:08:35.146600+0000","last_became_peered":"2023-11-28T11:08:35.146600+0000","last_unstale":"2023-11-28T11:08:39.546414+0000","last_undegraded":"2023-11-28T11:08:39.546414+0000","last_fullsized":"2023-11-28T11:08:39.546414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556350+0000","last_change":"2023-11-28T11:08:35.538328+0000","last_active":"2023-11-28T11:08:39.556350+0000","last_peered":"2023-11-28T11:08:39.556350+0000","last_clean":"2023-11-28T11:08:39.556350+0000","last_became_active":"2023-11-28T11:08:35.537863+0000","last_became_peered":"2023-11-28T11:08:35.537863+0000","last_unstale":"2023-11-28T11:08:39.556350+0000","last_undegraded":"2023-11-28T11:08:39.556350+0000","last_fullsized":"2023-11-28T11:08:39.556350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566546+0000","last_change":"2023-11-28T11:08:39.562424+0000","last_active":"2023-11-28T11:08:39.566546+0000","last_peered":"2023-11-28T11:08:39.566546+0000","last_clean":"2023-11-28T11:08:39.566546+0000","last_became_active":"2023-11-28T11:08:39.562065+0000","last_became_peered":"2023-11-28T11:08:39.562065+0000","last_unstale":"2023-11-28T11:08:39.566546+0000","last_undegraded":"2023-11-28T11:08:39.566546+0000","last_fullsized":"2023-11-28T11:08:39.566546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:39.095254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544783+0000","last_change":"2023-11-28T11:08:35.138855+0000","last_active":"2023-11-28T11:08:39.544783+0000","last_peered":"2023-11-28T11:08:39.544783+0000","last_clean":"2023-11-28T11:08:39.544783+0000","last_became_active":"2023-11-28T11:08:35.138526+0000","last_became_peered":"2023-11-28T11:08:35.138526+0000","last_unstale":"2023-11-28T11:08:39.544783+0000","last_undegraded":"2023-11-28T11:08:39.544783+0000","last_fullsized":"2023-11-28T11:08:39.544783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544513+0000","last_change":"2023-11-28T11:08:35.539629+0000","last_active":"2023-11-28T11:08:39.544513+0000","last_peered":"2023-11-28T11:08:39.544513+0000","last_clean":"2023-11-28T11:08:39.544513+0000","last_became_active":"2023-11-28T11:08:35.534961+0000","last_became_peered":"2023-11-28T11:08:35.534961+0000","last_unstale":"2023-11-28T11:08:39.544513+0000","last_undegraded":"2023-11-28T11:08:39.544513+0000","last_fullsized":"2023-11-28T11:08:39.544513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571346+0000","last_change":"2023-11-28T11:08:39.570212+0000","last_active":"2023-11-28T11:08:39.571346+0000","last_peered":"2023-11-28T11:08:39.571346+0000","last_clean":"2023-11-28T11:08:39.571346+0000","last_became_active":"2023-11-28T11:08:39.569900+0000","last_became_peered":"2023-11-28T11:08:39.569900+0000","last_unstale":"2023-11-28T11:08:39.571346+0000","last_undegraded":"2023-11-28T11:08:39.571346+0000","last_fullsized":"2023-11-28T11:08:39.571346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552849+0000","last_change":"2023-11-28T11:08:35.139415+0000","last_active":"2023-11-28T11:08:39.552849+0000","last_peered":"2023-11-28T11:08:39.552849+0000","last_clean":"2023-11-28T11:08:39.552849+0000","last_became_active":"2023-11-28T11:08:35.139152+0000","last_became_peered":"2023-11-28T11:08:35.139152+0000","last_unstale":"2023-11-28T11:08:39.552849+0000","last_undegraded":"2023-11-28T11:08:39.552849+0000","last_fullsized":"2023-11-28T11:08:39.552849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552466+0000","last_change":"2023-11-28T11:08:35.544319+0000","last_active":"2023-11-28T11:08:39.552466+0000","last_peered":"2023-11-28T11:08:39.552466+0000","last_clean":"2023-11-28T11:08:39.552466+0000","last_became_active":"2023-11-28T11:08:35.544032+0000","last_became_peered":"2023-11-28T11:08:35.544032+0000","last_unstale":"2023-11-28T11:08:39.552466+0000","last_undegraded":"2023-11-28T11:08:39.552466+0000","last_fullsized":"2023-11-28T11:08:39.552466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:36.189248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551454+0000","last_change":"2023-11-28T11:08:35.534208+0000","last_active":"2023-11-28T11:08:39.551454+0000","last_peered":"2023-11-28T11:08:39.551454+0000","last_clean":"2023-11-28T11:08:39.551454+0000","last_became_active":"2023-11-28T11:08:35.533929+0000","last_became_peered":"2023-11-28T11:08:35.533929+0000","last_unstale":"2023-11-28T11:08:39.551454+0000","last_undegraded":"2023-11-28T11:08:39.551454+0000","last_fullsized":"2023-11-28T11:08:39.551454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:44.139630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544658+0000","last_change":"2023-11-28T11:08:35.540664+0000","last_active":"2023-11-28T11:08:39.544658+0000","last_peered":"2023-11-28T11:08:39.544658+0000","last_clean":"2023-11-28T11:08:39.544658+0000","last_became_active":"2023-11-28T11:08:35.540243+0000","last_became_peered":"2023-11-28T11:08:35.540243+0000","last_unstale":"2023-11-28T11:08:39.544658+0000","last_undegraded":"2023-11-28T11:08:39.544658+0000","last_fullsized":"2023-11-28T11:08:39.544658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556410+0000","last_change":"2023-11-28T11:08:35.544372+0000","last_active":"2023-11-28T11:08:39.556410+0000","last_peered":"2023-11-28T11:08:39.556410+0000","last_clean":"2023-11-28T11:08:39.556410+0000","last_became_active":"2023-11-28T11:08:35.543867+0000","last_became_peered":"2023-11-28T11:08:35.543867+0000","last_unstale":"2023-11-28T11:08:39.556410+0000","last_undegraded":"2023-11-28T11:08:39.556410+0000","last_fullsized":"2023-11-28T11:08:39.556410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:30.327766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544832+0000","last_change":"2023-11-28T11:08:35.543075+0000","last_active":"2023-11-28T11:08:39.544832+0000","last_peered":"2023-11-28T11:08:39.544832+0000","last_clean":"2023-11-28T11:08:39.544832+0000","last_became_active":"2023-11-28T11:08:35.542645+0000","last_became_peered":"2023-11-28T11:08:35.542645+0000","last_unstale":"2023-11-28T11:08:39.544832+0000","last_undegraded":"2023-11-28T11:08:39.544832+0000","last_fullsized":"2023-11-28T11:08:39.544832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555411+0000","last_change":"2023-11-28T11:08:35.546225+0000","last_active":"2023-11-28T11:08:39.555411+0000","last_peered":"2023-11-28T11:08:39.555411+0000","last_clean":"2023-11-28T11:08:39.555411+0000","last_became_active":"2023-11-28T11:08:35.545970+0000","last_became_peered":"2023-11-28T11:08:35.545970+0000","last_unstale":"2023-11-28T11:08:39.555411+0000","last_undegraded":"2023-11-28T11:08:39.555411+0000","last_fullsized":"2023-11-28T11:08:39.555411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9248,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362592,"statfs":{"total":96636764160,"available":96627294208,"internally_reserved":0,"allocated":876544,"data_stored":699060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8060,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363780,"statfs":{"total":96636764160,"available":96628510720,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8128,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363712,"statfs":{"total":96636764160,"available":96628441088,"internally_reserved":0,"allocated":253952,"data_stored":103935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8064,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363776,"statfs":{"total":96636764160,"available":96628506624,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8132,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363708,"statfs":{"total":96636764160,"available":96628436992,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8212,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363628,"statfs":{"total":96636764160,"available":96628355072,"internally_reserved":0,"allocated":274432,"data_stored":108532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":105366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:57.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.679+0000 7fe1c825e700 1 Processor -- start 2023-11-28T11:08:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.679+0000 7fe1c825e700 1 -- start start 2023-11-28T11:08:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c825e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c00cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c825e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1c00b5170 0x7fe1c00d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c825e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c00d0c50 con 0x7fe1c0160e60 2023-11-28T11:08:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c825e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c00b8ca0 con 0x7fe1c00b5170 2023-11-28T11:08:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c825e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00cd760 con 0x7fe1c00b88d0 2023-11-28T11:08:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1c00b5170 0x7fe1c00d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c5ffa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c00cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1c00b5170 0x7fe1c00d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:36116/0 (socket says 172.21.15.83:36116) 2023-11-28T11:08:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c57f9700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fe1c0160e60 0x7fe1c00b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45324/0 (socket says 172.21.15.83:45324) 2023-11-28T11:08:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 -- 172.21.15.83:0/562045280 learned_addr learned my addr 172.21.15.83:0/562045280 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 -- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 msgr2=0x7fe1c00cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 --2- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c00cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 -- 172.21.15.83:0/562045280 >> v1:172.21.15.90:6789/0 conn(0x7fe1c0160e60 legacy=0x7fe1c00b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 -- 172.21.15.83:0/562045280 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00cda40 con 0x7fe1c00b5170 2023-11-28T11:08:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.680+0000 7fe1c67fb700 1 --2- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c00b5170 0x7fe1c00d1410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009e50 tx=0x7fe1b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6264ac63f922f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c4ff8700 1 -- 172.21.15.83:0/562045280 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800e070 con 0x7fe1c00b5170 2023-11-28T11:08:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c4ff8700 1 -- 172.21.15.83:0/562045280 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b800b910 con 0x7fe1c00b5170 2023-11-28T11:08:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c4ff8700 1 -- 172.21.15.83:0/562045280 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800bcc0 con 0x7fe1c00b5170 2023-11-28T11:08:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 -- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c00b5170 msgr2=0x7fe1c00d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 --2- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c00b5170 0x7fe1c00d1410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009e50 tx=0x7fe1b800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 -- 172.21.15.83:0/562045280 shutdown_connections 2023-11-28T11:08:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 --2- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c00b5170 0x7fe1c00d1410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 --2- 172.21.15.83:0/562045280 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c00cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 -- 172.21.15.83:0/562045280 >> 172.21.15.83:0/562045280 conn(0x7fe1c006d030 msgr2=0x7fe1c00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 -- 172.21.15.83:0/562045280 shutdown_connections 2023-11-28T11:08:57.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.681+0000 7fe1c825e700 1 -- 172.21.15.83:0/562045280 wait complete. 2023-11-28T11:08:57.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.682+0000 7fe1c825e700 1 Processor -- start 2023-11-28T11:08:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 -- start start 2023-11-28T11:08:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 0x7fe1c0155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c0160e60 0x7fe1c00cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c015e420 con 0x7fe1c00b5170 2023-11-28T11:08:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c015d7a0 con 0x7fe1c0160e60 2023-11-28T11:08:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c825e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c015da80 con 0x7fe1c00b88d0 2023-11-28T11:08:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c5ffa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 0x7fe1c0155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c5ffa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 0x7fe1c0155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49124/0 (socket says 172.21.15.83:49124) 2023-11-28T11:08:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.683+0000 7fe1c5ffa700 1 -- 172.21.15.83:0/2277652628 learned_addr learned my addr 172.21.15.83:0/2277652628 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c67fb700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c0160e60 0x7fe1c00cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c57f9700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c5ffa700 1 -- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 msgr2=0x7fe1c015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c5ffa700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c5ffa700 1 -- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c0160e60 msgr2=0x7fe1c00cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c5ffa700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c0160e60 0x7fe1c00cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c5ffa700 1 -- 172.21.15.83:0/2277652628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8009b30 con 0x7fe1c00b5170 2023-11-28T11:08:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c57f9700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c5ffa700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 0x7fe1c0155410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9b0 tx=0x7fe1bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc009710 con 0x7fe1c00b5170 2023-11-28T11:08:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00c850 con 0x7fe1c00b5170 2023-11-28T11:08:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c01612d0 con 0x7fe1c00b5170 2023-11-28T11:08:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00cbc0 con 0x7fe1c00b5170 2023-11-28T11:08:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.684+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c00d0ef0 con 0x7fe1c00b5170 2023-11-28T11:08:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.685+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c00cdd20 con 0x7fe1c00b5170 2023-11-28T11:08:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.685+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe1bc019030 con 0x7fe1c00b5170 2023-11-28T11:08:57.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.686+0000 7fe1b67fc700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1a40682f0 0x7fe1a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.686+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe1bc085f40 con 0x7fe1c00b5170 2023-11-28T11:08:57.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.686+0000 7fe1c57f9700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1a40682f0 0x7fe1a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.686+0000 7fe1c57f9700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1a40682f0 0x7fe1a406a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00ca010 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.690+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1bc0524b0 con 0x7fe1c00b5170 2023-11-28T11:08:57.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.869+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1c015dc40 con 0x7fe1a40682f0 2023-11-28T11:08:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.878+0000 7fe1b67fc700 1 -- 172.21.15.83:0/2277652628 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367512 (secure 0 0 0) 0x7fe1b0001c80 con 0x7fe1a40682f0 2023-11-28T11:08:57.885 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1a40682f0 msgr2=0x7fe1a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1a40682f0 0x7fe1a406a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00ca010 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 msgr2=0x7fe1c0155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 0x7fe1c0155410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9b0 tx=0x7fe1bc00ed70 comp rx=0 tx=0).stop 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 shutdown_connections 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1c0160e60 0x7fe1c00cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe1a40682f0 0x7fe1a406a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c00b5170 0x7fe1c0155410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 --2- 172.21.15.83:0/2277652628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1c00b88d0 0x7fe1c015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 >> 172.21.15.83:0/2277652628 conn(0x7fe1c006d030 msgr2=0x7fe1c00d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 shutdown_connections 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:57.882+0000 7fe1c825e700 1 -- 172.21.15.83:0/2277652628 wait complete. 2023-11-28T11:08:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:08:57.895 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-11-28T11:08:57.573389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67136,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907584,"statfs":{"total":773094113280,"available":773025366016,"internally_reserved":0,"allocated":3276800,"data_stored":2024910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1749,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002655"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542681+0000","last_change":"2023-11-28T11:08:35.535687+0000","last_active":"2023-11-28T11:08:39.542681+0000","last_peered":"2023-11-28T11:08:39.542681+0000","last_clean":"2023-11-28T11:08:39.542681+0000","last_became_active":"2023-11-28T11:08:35.535025+0000","last_became_peered":"2023-11-28T11:08:35.535025+0000","last_unstale":"2023-11-28T11:08:39.542681+0000","last_undegraded":"2023-11-28T11:08:39.542681+0000","last_fullsized":"2023-11-28T11:08:39.542681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:21.578670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546994+0000","last_change":"2023-11-28T11:08:35.534288+0000","last_active":"2023-11-28T11:08:39.546994+0000","last_peered":"2023-11-28T11:08:39.546994+0000","last_clean":"2023-11-28T11:08:39.546994+0000","last_became_active":"2023-11-28T11:08:35.533997+0000","last_became_peered":"2023-11-28T11:08:35.533997+0000","last_unstale":"2023-11-28T11:08:39.546994+0000","last_undegraded":"2023-11-28T11:08:39.546994+0000","last_fullsized":"2023-11-28T11:08:39.546994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556945+0000","last_change":"2023-11-28T11:08:35.553042+0000","last_active":"2023-11-28T11:08:39.556945+0000","last_peered":"2023-11-28T11:08:39.556945+0000","last_clean":"2023-11-28T11:08:39.556945+0000","last_became_active":"2023-11-28T11:08:35.552693+0000","last_became_peered":"2023-11-28T11:08:35.552693+0000","last_unstale":"2023-11-28T11:08:39.556945+0000","last_undegraded":"2023-11-28T11:08:39.556945+0000","last_fullsized":"2023-11-28T11:08:39.556945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:43.204644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551967+0000","last_change":"2023-11-28T11:08:35.538585+0000","last_active":"2023-11-28T11:08:39.551967+0000","last_peered":"2023-11-28T11:08:39.551967+0000","last_clean":"2023-11-28T11:08:39.551967+0000","last_became_active":"2023-11-28T11:08:35.538074+0000","last_became_peered":"2023-11-28T11:08:35.538074+0000","last_unstale":"2023-11-28T11:08:39.551967+0000","last_undegraded":"2023-11-28T11:08:39.551967+0000","last_fullsized":"2023-11-28T11:08:39.551967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:13.796245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556389+0000","last_change":"2023-11-28T11:08:35.548649+0000","last_active":"2023-11-28T11:08:39.556389+0000","last_peered":"2023-11-28T11:08:39.556389+0000","last_clean":"2023-11-28T11:08:39.556389+0000","last_became_active":"2023-11-28T11:08:35.546646+0000","last_became_peered":"2023-11-28T11:08:35.546646+0000","last_unstale":"2023-11-28T11:08:39.556389+0000","last_undegraded":"2023-11-28T11:08:39.556389+0000","last_fullsized":"2023-11-28T11:08:39.556389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556449+0000","last_change":"2023-11-28T11:08:35.545320+0000","last_active":"2023-11-28T11:08:39.556449+0000","last_peered":"2023-11-28T11:08:39.556449+0000","last_clean":"2023-11-28T11:08:39.556449+0000","last_became_active":"2023-11-28T11:08:35.545051+0000","last_became_peered":"2023-11-28T11:08:35.545051+0000","last_unstale":"2023-11-28T11:08:39.556449+0000","last_undegraded":"2023-11-28T11:08:39.556449+0000","last_fullsized":"2023-11-28T11:08:39.556449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549589+0000","last_change":"2023-11-28T11:08:35.543641+0000","last_active":"2023-11-28T11:08:39.549589+0000","last_peered":"2023-11-28T11:08:39.549589+0000","last_clean":"2023-11-28T11:08:39.549589+0000","last_became_active":"2023-11-28T11:08:35.543407+0000","last_became_peered":"2023-11-28T11:08:35.543407+0000","last_unstale":"2023-11-28T11:08:39.549589+0000","last_undegraded":"2023-11-28T11:08:39.549589+0000","last_fullsized":"2023-11-28T11:08:39.549589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546097+0000","last_change":"2023-11-28T11:08:35.538922+0000","last_active":"2023-11-28T11:08:39.546097+0000","last_peered":"2023-11-28T11:08:39.546097+0000","last_clean":"2023-11-28T11:08:39.546097+0000","last_became_active":"2023-11-28T11:08:35.538455+0000","last_became_peered":"2023-11-28T11:08:35.538455+0000","last_unstale":"2023-11-28T11:08:39.546097+0000","last_undegraded":"2023-11-28T11:08:39.546097+0000","last_fullsized":"2023-11-28T11:08:39.546097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549757+0000","last_change":"2023-11-28T11:08:35.550345+0000","last_active":"2023-11-28T11:08:39.549757+0000","last_peered":"2023-11-28T11:08:39.549757+0000","last_clean":"2023-11-28T11:08:39.549757+0000","last_became_active":"2023-11-28T11:08:35.546222+0000","last_became_peered":"2023-11-28T11:08:35.546222+0000","last_unstale":"2023-11-28T11:08:39.549757+0000","last_undegraded":"2023-11-28T11:08:39.549757+0000","last_fullsized":"2023-11-28T11:08:39.549757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546977+0000","last_change":"2023-11-28T11:08:35.550152+0000","last_active":"2023-11-28T11:08:39.546977+0000","last_peered":"2023-11-28T11:08:39.546977+0000","last_clean":"2023-11-28T11:08:39.546977+0000","last_became_active":"2023-11-28T11:08:35.549618+0000","last_became_peered":"2023-11-28T11:08:35.549618+0000","last_unstale":"2023-11-28T11:08:39.546977+0000","last_undegraded":"2023-11-28T11:08:39.546977+0000","last_fullsized":"2023-11-28T11:08:39.546977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556901+0000","last_change":"2023-11-28T11:08:35.544292+0000","last_active":"2023-11-28T11:08:39.556901+0000","last_peered":"2023-11-28T11:08:39.556901+0000","last_clean":"2023-11-28T11:08:39.556901+0000","last_became_active":"2023-11-28T11:08:35.543905+0000","last_became_peered":"2023-11-28T11:08:35.543905+0000","last_unstale":"2023-11-28T11:08:39.556901+0000","last_undegraded":"2023-11-28T11:08:39.556901+0000","last_fullsized":"2023-11-28T11:08:39.556901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:46.079543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542701+0000","last_change":"2023-11-28T11:08:35.548959+0000","last_active":"2023-11-28T11:08:39.542701+0000","last_peered":"2023-11-28T11:08:39.542701+0000","last_clean":"2023-11-28T11:08:39.542701+0000","last_became_active":"2023-11-28T11:08:35.548707+0000","last_became_peered":"2023-11-28T11:08:35.548707+0000","last_unstale":"2023-11-28T11:08:39.542701+0000","last_undegraded":"2023-11-28T11:08:39.542701+0000","last_fullsized":"2023-11-28T11:08:39.542701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:41.426965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542575+0000","last_change":"2023-11-28T11:08:35.534783+0000","last_active":"2023-11-28T11:08:39.542575+0000","last_peered":"2023-11-28T11:08:39.542575+0000","last_clean":"2023-11-28T11:08:39.542575+0000","last_became_active":"2023-11-28T11:08:35.534520+0000","last_became_peered":"2023-11-28T11:08:35.534520+0000","last_unstale":"2023-11-28T11:08:39.542575+0000","last_undegraded":"2023-11-28T11:08:39.542575+0000","last_fullsized":"2023-11-28T11:08:39.542575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:56.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546312+0000","last_change":"2023-11-28T11:08:35.538322+0000","last_active":"2023-11-28T11:08:39.546312+0000","last_peered":"2023-11-28T11:08:39.546312+0000","last_clean":"2023-11-28T11:08:39.546312+0000","last_became_active":"2023-11-28T11:08:35.538000+0000","last_became_peered":"2023-11-28T11:08:35.538000+0000","last_unstale":"2023-11-28T11:08:39.546312+0000","last_undegraded":"2023-11-28T11:08:39.546312+0000","last_fullsized":"2023-11-28T11:08:39.546312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549399+0000","last_change":"2023-11-28T11:08:35.544771+0000","last_active":"2023-11-28T11:08:39.549399+0000","last_peered":"2023-11-28T11:08:39.549399+0000","last_clean":"2023-11-28T11:08:39.549399+0000","last_became_active":"2023-11-28T11:08:35.544525+0000","last_became_peered":"2023-11-28T11:08:35.544525+0000","last_unstale":"2023-11-28T11:08:39.549399+0000","last_undegraded":"2023-11-28T11:08:39.549399+0000","last_fullsized":"2023-11-28T11:08:39.549399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552566+0000","last_change":"2023-11-28T11:08:35.547686+0000","last_active":"2023-11-28T11:08:39.552566+0000","last_peered":"2023-11-28T11:08:39.552566+0000","last_clean":"2023-11-28T11:08:39.552566+0000","last_became_active":"2023-11-28T11:08:35.547411+0000","last_became_peered":"2023-11-28T11:08:35.547411+0000","last_unstale":"2023-11-28T11:08:39.552566+0000","last_undegraded":"2023-11-28T11:08:39.552566+0000","last_fullsized":"2023-11-28T11:08:39.552566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549830+0000","last_change":"2023-11-28T11:08:35.551541+0000","last_active":"2023-11-28T11:08:39.549830+0000","last_peered":"2023-11-28T11:08:39.549830+0000","last_clean":"2023-11-28T11:08:39.549830+0000","last_became_active":"2023-11-28T11:08:35.546691+0000","last_became_peered":"2023-11-28T11:08:35.546691+0000","last_unstale":"2023-11-28T11:08:39.549830+0000","last_undegraded":"2023-11-28T11:08:39.549830+0000","last_fullsized":"2023-11-28T11:08:39.549830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556352+0000","last_change":"2023-11-28T11:08:35.541127+0000","last_active":"2023-11-28T11:08:39.556352+0000","last_peered":"2023-11-28T11:08:39.556352+0000","last_clean":"2023-11-28T11:08:39.556352+0000","last_became_active":"2023-11-28T11:08:35.540800+0000","last_became_peered":"2023-11-28T11:08:35.540800+0000","last_unstale":"2023-11-28T11:08:39.556352+0000","last_undegraded":"2023-11-28T11:08:39.556352+0000","last_fullsized":"2023-11-28T11:08:39.556352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.543580+0000","last_change":"2023-11-28T11:08:35.546695+0000","last_active":"2023-11-28T11:08:39.543580+0000","last_peered":"2023-11-28T11:08:39.543580+0000","last_clean":"2023-11-28T11:08:39.543580+0000","last_became_active":"2023-11-28T11:08:35.546438+0000","last_became_peered":"2023-11-28T11:08:35.546438+0000","last_unstale":"2023-11-28T11:08:39.543580+0000","last_undegraded":"2023-11-28T11:08:39.543580+0000","last_fullsized":"2023-11-28T11:08:39.543580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:37.007416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556699+0000","last_change":"2023-11-28T11:08:35.548019+0000","last_active":"2023-11-28T11:08:39.556699+0000","last_peered":"2023-11-28T11:08:39.556699+0000","last_clean":"2023-11-28T11:08:39.556699+0000","last_became_active":"2023-11-28T11:08:35.541157+0000","last_became_peered":"2023-11-28T11:08:35.541157+0000","last_unstale":"2023-11-28T11:08:39.556699+0000","last_undegraded":"2023-11-28T11:08:39.556699+0000","last_fullsized":"2023-11-28T11:08:39.556699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:40.346506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551701+0000","last_change":"2023-11-28T11:08:35.540145+0000","last_active":"2023-11-28T11:08:39.551701+0000","last_peered":"2023-11-28T11:08:39.551701+0000","last_clean":"2023-11-28T11:08:39.551701+0000","last_became_active":"2023-11-28T11:08:35.539880+0000","last_became_peered":"2023-11-28T11:08:35.539880+0000","last_unstale":"2023-11-28T11:08:39.551701+0000","last_undegraded":"2023-11-28T11:08:39.551701+0000","last_fullsized":"2023-11-28T11:08:39.551701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549283+0000","last_change":"2023-11-28T11:08:35.546609+0000","last_active":"2023-11-28T11:08:39.549283+0000","last_peered":"2023-11-28T11:08:39.549283+0000","last_clean":"2023-11-28T11:08:39.549283+0000","last_became_active":"2023-11-28T11:08:35.545637+0000","last_became_peered":"2023-11-28T11:08:35.545637+0000","last_unstale":"2023-11-28T11:08:39.549283+0000","last_undegraded":"2023-11-28T11:08:39.549283+0000","last_fullsized":"2023-11-28T11:08:39.549283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544746+0000","last_change":"2023-11-28T11:08:35.540494+0000","last_active":"2023-11-28T11:08:39.544746+0000","last_peered":"2023-11-28T11:08:39.544746+0000","last_clean":"2023-11-28T11:08:39.544746+0000","last_became_active":"2023-11-28T11:08:35.539936+0000","last_became_peered":"2023-11-28T11:08:35.539936+0000","last_unstale":"2023-11-28T11:08:39.544746+0000","last_undegraded":"2023-11-28T11:08:39.544746+0000","last_fullsized":"2023-11-28T11:08:39.544746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550772+0000","last_change":"2023-11-28T11:08:35.545757+0000","last_active":"2023-11-28T11:08:39.550772+0000","last_peered":"2023-11-28T11:08:39.550772+0000","last_clean":"2023-11-28T11:08:39.550772+0000","last_became_active":"2023-11-28T11:08:35.545459+0000","last_became_peered":"2023-11-28T11:08:35.545459+0000","last_unstale":"2023-11-28T11:08:39.550772+0000","last_undegraded":"2023-11-28T11:08:39.550772+0000","last_fullsized":"2023-11-28T11:08:39.550772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556095+0000","last_change":"2023-11-28T11:08:35.540552+0000","last_active":"2023-11-28T11:08:39.556095+0000","last_peered":"2023-11-28T11:08:39.556095+0000","last_clean":"2023-11-28T11:08:39.556095+0000","last_became_active":"2023-11-28T11:08:35.539972+0000","last_became_peered":"2023-11-28T11:08:35.539972+0000","last_unstale":"2023-11-28T11:08:39.556095+0000","last_undegraded":"2023-11-28T11:08:39.556095+0000","last_fullsized":"2023-11-28T11:08:39.556095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556230+0000","last_change":"2023-11-28T11:08:35.536499+0000","last_active":"2023-11-28T11:08:39.556230+0000","last_peered":"2023-11-28T11:08:39.556230+0000","last_clean":"2023-11-28T11:08:39.556230+0000","last_became_active":"2023-11-28T11:08:35.536308+0000","last_became_peered":"2023-11-28T11:08:35.536308+0000","last_unstale":"2023-11-28T11:08:39.556230+0000","last_undegraded":"2023-11-28T11:08:39.556230+0000","last_fullsized":"2023-11-28T11:08:39.556230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:34.912505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546678+0000","last_change":"2023-11-28T11:08:35.542161+0000","last_active":"2023-11-28T11:08:39.546678+0000","last_peered":"2023-11-28T11:08:39.546678+0000","last_clean":"2023-11-28T11:08:39.546678+0000","last_became_active":"2023-11-28T11:08:35.541331+0000","last_became_peered":"2023-11-28T11:08:35.541331+0000","last_unstale":"2023-11-28T11:08:39.546678+0000","last_undegraded":"2023-11-28T11:08:39.546678+0000","last_fullsized":"2023-11-28T11:08:39.546678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570777+0000","last_change":"2023-11-28T11:08:39.566823+0000","last_active":"2023-11-28T11:08:39.570777+0000","last_peered":"2023-11-28T11:08:39.570777+0000","last_clean":"2023-11-28T11:08:39.570777+0000","last_became_active":"2023-11-28T11:08:39.563462+0000","last_became_peered":"2023-11-28T11:08:39.563462+0000","last_unstale":"2023-11-28T11:08:39.570777+0000","last_undegraded":"2023-11-28T11:08:39.570777+0000","last_fullsized":"2023-11-28T11:08:39.570777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:25.821485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556318+0000","last_change":"2023-11-28T11:08:35.144920+0000","last_active":"2023-11-28T11:08:39.556318+0000","last_peered":"2023-11-28T11:08:39.556318+0000","last_clean":"2023-11-28T11:08:39.556318+0000","last_became_active":"2023-11-28T11:08:35.144360+0000","last_became_peered":"2023-11-28T11:08:35.144360+0000","last_unstale":"2023-11-28T11:08:39.556318+0000","last_undegraded":"2023-11-28T11:08:39.556318+0000","last_fullsized":"2023-11-28T11:08:39.556318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544509+0000","last_change":"2023-11-28T11:08:35.537580+0000","last_active":"2023-11-28T11:08:39.544509+0000","last_peered":"2023-11-28T11:08:39.544509+0000","last_clean":"2023-11-28T11:08:39.544509+0000","last_became_active":"2023-11-28T11:08:35.537300+0000","last_became_peered":"2023-11-28T11:08:35.537300+0000","last_unstale":"2023-11-28T11:08:39.544509+0000","last_undegraded":"2023-11-28T11:08:39.544509+0000","last_fullsized":"2023-11-28T11:08:39.544509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:23.470863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573888+0000","last_change":"2023-11-28T11:08:39.565761+0000","last_active":"2023-11-28T11:08:39.573888+0000","last_peered":"2023-11-28T11:08:39.573888+0000","last_clean":"2023-11-28T11:08:39.573888+0000","last_became_active":"2023-11-28T11:08:39.565387+0000","last_became_peered":"2023-11-28T11:08:39.565387+0000","last_unstale":"2023-11-28T11:08:39.573888+0000","last_undegraded":"2023-11-28T11:08:39.573888+0000","last_fullsized":"2023-11-28T11:08:39.573888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:29.648074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552645+0000","last_change":"2023-11-28T11:08:35.141536+0000","last_active":"2023-11-28T11:08:39.552645+0000","last_peered":"2023-11-28T11:08:39.552645+0000","last_clean":"2023-11-28T11:08:39.552645+0000","last_became_active":"2023-11-28T11:08:35.141259+0000","last_became_peered":"2023-11-28T11:08:35.141259+0000","last_unstale":"2023-11-28T11:08:39.552645+0000","last_undegraded":"2023-11-28T11:08:39.552645+0000","last_fullsized":"2023-11-28T11:08:39.552645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551925+0000","last_change":"2023-11-28T11:08:35.551235+0000","last_active":"2023-11-28T11:08:39.551925+0000","last_peered":"2023-11-28T11:08:39.551925+0000","last_clean":"2023-11-28T11:08:39.551925+0000","last_became_active":"2023-11-28T11:08:35.550945+0000","last_became_peered":"2023-11-28T11:08:35.550945+0000","last_unstale":"2023-11-28T11:08:39.551925+0000","last_undegraded":"2023-11-28T11:08:39.551925+0000","last_fullsized":"2023-11-28T11:08:39.551925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.576247+0000","last_change":"2023-11-28T11:08:39.574297+0000","last_active":"2023-11-28T11:08:39.576247+0000","last_peered":"2023-11-28T11:08:39.576247+0000","last_clean":"2023-11-28T11:08:39.576247+0000","last_became_active":"2023-11-28T11:08:39.573836+0000","last_became_peered":"2023-11-28T11:08:39.573836+0000","last_unstale":"2023-11-28T11:08:39.576247+0000","last_undegraded":"2023-11-28T11:08:39.576247+0000","last_fullsized":"2023-11-28T11:08:39.576247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546095+0000","last_change":"2023-11-28T11:08:35.140807+0000","last_active":"2023-11-28T11:08:39.546095+0000","last_peered":"2023-11-28T11:08:39.546095+0000","last_clean":"2023-11-28T11:08:39.546095+0000","last_became_active":"2023-11-28T11:08:35.140551+0000","last_became_peered":"2023-11-28T11:08:35.140551+0000","last_unstale":"2023-11-28T11:08:39.546095+0000","last_undegraded":"2023-11-28T11:08:39.546095+0000","last_fullsized":"2023-11-28T11:08:39.546095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553480+0000","last_change":"2023-11-28T11:08:35.540497+0000","last_active":"2023-11-28T11:08:39.553480+0000","last_peered":"2023-11-28T11:08:39.553480+0000","last_clean":"2023-11-28T11:08:39.553480+0000","last_became_active":"2023-11-28T11:08:35.540154+0000","last_became_peered":"2023-11-28T11:08:35.540154+0000","last_unstale":"2023-11-28T11:08:39.553480+0000","last_undegraded":"2023-11-28T11:08:39.553480+0000","last_fullsized":"2023-11-28T11:08:39.553480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:43.354051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.574001+0000","last_change":"2023-11-28T11:08:39.568636+0000","last_active":"2023-11-28T11:08:39.574001+0000","last_peered":"2023-11-28T11:08:39.574001+0000","last_clean":"2023-11-28T11:08:39.574001+0000","last_became_active":"2023-11-28T11:08:39.568255+0000","last_became_peered":"2023-11-28T11:08:39.568255+0000","last_unstale":"2023-11-28T11:08:39.574001+0000","last_undegraded":"2023-11-28T11:08:39.574001+0000","last_fullsized":"2023-11-28T11:08:39.574001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549903+0000","last_change":"2023-11-28T11:08:35.139702+0000","last_active":"2023-11-28T11:08:39.549903+0000","last_peered":"2023-11-28T11:08:39.549903+0000","last_clean":"2023-11-28T11:08:39.549903+0000","last_became_active":"2023-11-28T11:08:35.139379+0000","last_became_peered":"2023-11-28T11:08:35.139379+0000","last_unstale":"2023-11-28T11:08:39.549903+0000","last_undegraded":"2023-11-28T11:08:39.549903+0000","last_fullsized":"2023-11-28T11:08:39.549903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546007+0000","last_change":"2023-11-28T11:08:35.532733+0000","last_active":"2023-11-28T11:08:39.546007+0000","last_peered":"2023-11-28T11:08:39.546007+0000","last_clean":"2023-11-28T11:08:39.546007+0000","last_became_active":"2023-11-28T11:08:35.532390+0000","last_became_peered":"2023-11-28T11:08:35.532390+0000","last_unstale":"2023-11-28T11:08:39.546007+0000","last_undegraded":"2023-11-28T11:08:39.546007+0000","last_fullsized":"2023-11-28T11:08:39.546007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573440+0000","last_change":"2023-11-28T11:08:39.568834+0000","last_active":"2023-11-28T11:08:39.573440+0000","last_peered":"2023-11-28T11:08:39.573440+0000","last_clean":"2023-11-28T11:08:39.573440+0000","last_became_active":"2023-11-28T11:08:39.568153+0000","last_became_peered":"2023-11-28T11:08:39.568153+0000","last_unstale":"2023-11-28T11:08:39.573440+0000","last_undegraded":"2023-11-28T11:08:39.573440+0000","last_fullsized":"2023-11-28T11:08:39.573440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546860+0000","last_change":"2023-11-28T11:08:35.140772+0000","last_active":"2023-11-28T11:08:39.546860+0000","last_peered":"2023-11-28T11:08:39.546860+0000","last_clean":"2023-11-28T11:08:39.546860+0000","last_became_active":"2023-11-28T11:08:35.140418+0000","last_became_peered":"2023-11-28T11:08:35.140418+0000","last_unstale":"2023-11-28T11:08:39.546860+0000","last_undegraded":"2023-11-28T11:08:39.546860+0000","last_fullsized":"2023-11-28T11:08:39.546860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:36.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544363+0000","last_change":"2023-11-28T11:08:35.544651+0000","last_active":"2023-11-28T11:08:39.544363+0000","last_peered":"2023-11-28T11:08:39.544363+0000","last_clean":"2023-11-28T11:08:39.544363+0000","last_became_active":"2023-11-28T11:08:35.541280+0000","last_became_peered":"2023-11-28T11:08:35.541280+0000","last_unstale":"2023-11-28T11:08:39.544363+0000","last_undegraded":"2023-11-28T11:08:39.544363+0000","last_fullsized":"2023-11-28T11:08:39.544363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568222+0000","last_change":"2023-11-28T11:08:39.562858+0000","last_active":"2023-11-28T11:08:39.568222+0000","last_peered":"2023-11-28T11:08:39.568222+0000","last_clean":"2023-11-28T11:08:39.568222+0000","last_became_active":"2023-11-28T11:08:39.562273+0000","last_became_peered":"2023-11-28T11:08:39.562273+0000","last_unstale":"2023-11-28T11:08:39.568222+0000","last_undegraded":"2023-11-28T11:08:39.568222+0000","last_fullsized":"2023-11-28T11:08:39.568222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:39.831045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552124+0000","last_change":"2023-11-28T11:08:35.144557+0000","last_active":"2023-11-28T11:08:39.552124+0000","last_peered":"2023-11-28T11:08:39.552124+0000","last_clean":"2023-11-28T11:08:39.552124+0000","last_became_active":"2023-11-28T11:08:35.144222+0000","last_became_peered":"2023-11-28T11:08:35.144222+0000","last_unstale":"2023-11-28T11:08:39.552124+0000","last_undegraded":"2023-11-28T11:08:39.552124+0000","last_fullsized":"2023-11-28T11:08:39.552124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552994+0000","last_change":"2023-11-28T11:08:35.543394+0000","last_active":"2023-11-28T11:08:39.552994+0000","last_peered":"2023-11-28T11:08:39.552994+0000","last_clean":"2023-11-28T11:08:39.552994+0000","last_became_active":"2023-11-28T11:08:35.543099+0000","last_became_peered":"2023-11-28T11:08:35.543099+0000","last_unstale":"2023-11-28T11:08:39.552994+0000","last_undegraded":"2023-11-28T11:08:39.552994+0000","last_fullsized":"2023-11-28T11:08:39.552994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571358+0000","last_change":"2023-11-28T11:08:39.570301+0000","last_active":"2023-11-28T11:08:39.571358+0000","last_peered":"2023-11-28T11:08:39.571358+0000","last_clean":"2023-11-28T11:08:39.571358+0000","last_became_active":"2023-11-28T11:08:39.569828+0000","last_became_peered":"2023-11-28T11:08:39.569828+0000","last_unstale":"2023-11-28T11:08:39.571358+0000","last_undegraded":"2023-11-28T11:08:39.571358+0000","last_fullsized":"2023-11-28T11:08:39.571358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.056449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:44.461165+0000","last_change":"2023-11-28T11:08:35.147185+0000","last_active":"2023-11-28T11:08:44.461165+0000","last_peered":"2023-11-28T11:08:44.461165+0000","last_clean":"2023-11-28T11:08:44.461165+0000","last_became_active":"2023-11-28T11:08:35.146285+0000","last_became_peered":"2023-11-28T11:08:35.146285+0000","last_unstale":"2023-11-28T11:08:44.461165+0000","last_undegraded":"2023-11-28T11:08:44.461165+0000","last_fullsized":"2023-11-28T11:08:44.461165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556565+0000","last_change":"2023-11-28T11:08:35.546895+0000","last_active":"2023-11-28T11:08:39.556565+0000","last_peered":"2023-11-28T11:08:39.556565+0000","last_clean":"2023-11-28T11:08:39.556565+0000","last_became_active":"2023-11-28T11:08:35.538447+0000","last_became_peered":"2023-11-28T11:08:35.538447+0000","last_unstale":"2023-11-28T11:08:39.556565+0000","last_undegraded":"2023-11-28T11:08:39.556565+0000","last_fullsized":"2023-11-28T11:08:39.556565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:28.436759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564343+0000","last_change":"2023-11-28T11:08:39.560554+0000","last_active":"2023-11-28T11:08:39.564343+0000","last_peered":"2023-11-28T11:08:39.564343+0000","last_clean":"2023-11-28T11:08:39.564343+0000","last_became_active":"2023-11-28T11:08:39.560032+0000","last_became_peered":"2023-11-28T11:08:39.560032+0000","last_unstale":"2023-11-28T11:08:39.564343+0000","last_undegraded":"2023-11-28T11:08:39.564343+0000","last_fullsized":"2023-11-28T11:08:39.564343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551526+0000","last_change":"2023-11-28T11:08:35.146846+0000","last_active":"2023-11-28T11:08:39.551526+0000","last_peered":"2023-11-28T11:08:39.551526+0000","last_clean":"2023-11-28T11:08:39.551526+0000","last_became_active":"2023-11-28T11:08:35.146565+0000","last_became_peered":"2023-11-28T11:08:35.146565+0000","last_unstale":"2023-11-28T11:08:39.551526+0000","last_undegraded":"2023-11-28T11:08:39.551526+0000","last_fullsized":"2023-11-28T11:08:39.551526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545959+0000","last_change":"2023-11-28T11:08:35.537655+0000","last_active":"2023-11-28T11:08:39.545959+0000","last_peered":"2023-11-28T11:08:39.545959+0000","last_clean":"2023-11-28T11:08:39.545959+0000","last_became_active":"2023-11-28T11:08:35.537401+0000","last_became_peered":"2023-11-28T11:08:35.537401+0000","last_unstale":"2023-11-28T11:08:39.545959+0000","last_undegraded":"2023-11-28T11:08:39.545959+0000","last_fullsized":"2023-11-28T11:08:39.545959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.559490+0000","last_change":"2023-11-28T11:08:39.558756+0000","last_active":"2023-11-28T11:08:39.559490+0000","last_peered":"2023-11-28T11:08:39.559490+0000","last_clean":"2023-11-28T11:08:39.559490+0000","last_became_active":"2023-11-28T11:08:39.558382+0000","last_became_peered":"2023-11-28T11:08:39.558382+0000","last_unstale":"2023-11-28T11:08:39.559490+0000","last_undegraded":"2023-11-28T11:08:39.559490+0000","last_fullsized":"2023-11-28T11:08:39.559490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:08.685769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549379+0000","last_change":"2023-11-28T11:08:35.143155+0000","last_active":"2023-11-28T11:08:39.549379+0000","last_peered":"2023-11-28T11:08:39.549379+0000","last_clean":"2023-11-28T11:08:39.549379+0000","last_became_active":"2023-11-28T11:08:35.142856+0000","last_became_peered":"2023-11-28T11:08:35.142856+0000","last_unstale":"2023-11-28T11:08:39.549379+0000","last_undegraded":"2023-11-28T11:08:39.549379+0000","last_fullsized":"2023-11-28T11:08:39.549379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546595+0000","last_change":"2023-11-28T11:08:35.550010+0000","last_active":"2023-11-28T11:08:39.546595+0000","last_peered":"2023-11-28T11:08:39.546595+0000","last_clean":"2023-11-28T11:08:39.546595+0000","last_became_active":"2023-11-28T11:08:35.548764+0000","last_became_peered":"2023-11-28T11:08:35.548764+0000","last_unstale":"2023-11-28T11:08:39.546595+0000","last_undegraded":"2023-11-28T11:08:39.546595+0000","last_fullsized":"2023-11-28T11:08:39.546595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:52.542003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568175+0000","last_change":"2023-11-28T11:08:39.562607+0000","last_active":"2023-11-28T11:08:39.568175+0000","last_peered":"2023-11-28T11:08:39.568175+0000","last_clean":"2023-11-28T11:08:39.568175+0000","last_became_active":"2023-11-28T11:08:39.562144+0000","last_became_peered":"2023-11-28T11:08:39.562144+0000","last_unstale":"2023-11-28T11:08:39.568175+0000","last_undegraded":"2023-11-28T11:08:39.568175+0000","last_fullsized":"2023-11-28T11:08:39.568175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:43:13.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549488+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.549488+0000","last_peered":"2023-11-28T11:08:39.549488+0000","last_clean":"2023-11-28T11:08:39.549488+0000","last_became_active":"2023-11-28T11:08:35.146468+0000","last_became_peered":"2023-11-28T11:08:35.146468+0000","last_unstale":"2023-11-28T11:08:39.549488+0000","last_undegraded":"2023-11-28T11:08:39.549488+0000","last_fullsized":"2023-11-28T11:08:39.549488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:42.348852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552146+0000","last_change":"2023-11-28T11:08:35.545831+0000","last_active":"2023-11-28T11:08:39.552146+0000","last_peered":"2023-11-28T11:08:39.552146+0000","last_clean":"2023-11-28T11:08:39.552146+0000","last_became_active":"2023-11-28T11:08:35.544509+0000","last_became_peered":"2023-11-28T11:08:35.544509+0000","last_unstale":"2023-11-28T11:08:39.552146+0000","last_undegraded":"2023-11-28T11:08:39.552146+0000","last_fullsized":"2023-11-28T11:08:39.552146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.564184+0000","last_change":"2023-11-28T11:08:39.560957+0000","last_active":"2023-11-28T11:08:39.564184+0000","last_peered":"2023-11-28T11:08:39.564184+0000","last_clean":"2023-11-28T11:08:39.564184+0000","last_became_active":"2023-11-28T11:08:39.560011+0000","last_became_peered":"2023-11-28T11:08:39.560011+0000","last_unstale":"2023-11-28T11:08:39.564184+0000","last_undegraded":"2023-11-28T11:08:39.564184+0000","last_fullsized":"2023-11-28T11:08:39.564184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551361+0000","last_change":"2023-11-28T11:08:35.137486+0000","last_active":"2023-11-28T11:08:39.551361+0000","last_peered":"2023-11-28T11:08:39.551361+0000","last_clean":"2023-11-28T11:08:39.551361+0000","last_became_active":"2023-11-28T11:08:35.137192+0000","last_became_peered":"2023-11-28T11:08:35.137192+0000","last_unstale":"2023-11-28T11:08:39.551361+0000","last_undegraded":"2023-11-28T11:08:39.551361+0000","last_fullsized":"2023-11-28T11:08:39.551361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:09.971307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545264+0000","last_change":"2023-11-28T11:08:35.543758+0000","last_active":"2023-11-28T11:08:39.545264+0000","last_peered":"2023-11-28T11:08:39.545264+0000","last_clean":"2023-11-28T11:08:39.545264+0000","last_became_active":"2023-11-28T11:08:35.543535+0000","last_became_peered":"2023-11-28T11:08:35.543535+0000","last_unstale":"2023-11-28T11:08:39.545264+0000","last_undegraded":"2023-11-28T11:08:39.545264+0000","last_fullsized":"2023-11-28T11:08:39.545264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570639+0000","last_change":"2023-11-28T11:08:39.567742+0000","last_active":"2023-11-28T11:08:39.570639+0000","last_peered":"2023-11-28T11:08:39.570639+0000","last_clean":"2023-11-28T11:08:39.570639+0000","last_became_active":"2023-11-28T11:08:39.567313+0000","last_became_peered":"2023-11-28T11:08:39.567313+0000","last_unstale":"2023-11-28T11:08:39.570639+0000","last_undegraded":"2023-11-28T11:08:39.570639+0000","last_fullsized":"2023-11-28T11:08:39.570639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552255+0000","last_change":"2023-11-28T11:08:35.142255+0000","last_active":"2023-11-28T11:08:39.552255+0000","last_peered":"2023-11-28T11:08:39.552255+0000","last_clean":"2023-11-28T11:08:39.552255+0000","last_became_active":"2023-11-28T11:08:35.140243+0000","last_became_peered":"2023-11-28T11:08:35.140243+0000","last_unstale":"2023-11-28T11:08:39.552255+0000","last_undegraded":"2023-11-28T11:08:39.552255+0000","last_fullsized":"2023-11-28T11:08:39.552255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:43.134431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546061+0000","last_change":"2023-11-28T11:08:35.540595+0000","last_active":"2023-11-28T11:08:39.546061+0000","last_peered":"2023-11-28T11:08:39.546061+0000","last_clean":"2023-11-28T11:08:39.546061+0000","last_became_active":"2023-11-28T11:08:35.540303+0000","last_became_peered":"2023-11-28T11:08:35.540303+0000","last_unstale":"2023-11-28T11:08:39.546061+0000","last_undegraded":"2023-11-28T11:08:39.546061+0000","last_fullsized":"2023-11-28T11:08:39.546061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562667+0000","last_change":"2023-11-28T11:08:39.560601+0000","last_active":"2023-11-28T11:08:39.562667+0000","last_peered":"2023-11-28T11:08:39.562667+0000","last_clean":"2023-11-28T11:08:39.562667+0000","last_became_active":"2023-11-28T11:08:39.560147+0000","last_became_peered":"2023-11-28T11:08:39.560147+0000","last_unstale":"2023-11-28T11:08:39.562667+0000","last_undegraded":"2023-11-28T11:08:39.562667+0000","last_fullsized":"2023-11-28T11:08:39.562667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:39.607355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552185+0000","last_change":"2023-11-28T11:08:35.142804+0000","last_active":"2023-11-28T11:08:39.552185+0000","last_peered":"2023-11-28T11:08:39.552185+0000","last_clean":"2023-11-28T11:08:39.552185+0000","last_became_active":"2023-11-28T11:08:35.142407+0000","last_became_peered":"2023-11-28T11:08:35.142407+0000","last_unstale":"2023-11-28T11:08:39.552185+0000","last_undegraded":"2023-11-28T11:08:39.552185+0000","last_fullsized":"2023-11-28T11:08:39.552185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:12.791046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560104+0000","last_change":"2023-11-28T11:08:35.543448+0000","last_active":"2023-11-28T11:08:39.560104+0000","last_peered":"2023-11-28T11:08:39.560104+0000","last_clean":"2023-11-28T11:08:39.560104+0000","last_became_active":"2023-11-28T11:08:35.541094+0000","last_became_peered":"2023-11-28T11:08:35.541094+0000","last_unstale":"2023-11-28T11:08:39.560104+0000","last_undegraded":"2023-11-28T11:08:39.560104+0000","last_fullsized":"2023-11-28T11:08:39.560104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571651+0000","last_change":"2023-11-28T11:08:39.562935+0000","last_active":"2023-11-28T11:08:39.571651+0000","last_peered":"2023-11-28T11:08:39.571651+0000","last_clean":"2023-11-28T11:08:39.571651+0000","last_became_active":"2023-11-28T11:08:39.562529+0000","last_became_peered":"2023-11-28T11:08:39.562529+0000","last_unstale":"2023-11-28T11:08:39.571651+0000","last_undegraded":"2023-11-28T11:08:39.571651+0000","last_fullsized":"2023-11-28T11:08:39.571651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:31.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552244+0000","last_change":"2023-11-28T11:08:35.143035+0000","last_active":"2023-11-28T11:08:39.552244+0000","last_peered":"2023-11-28T11:08:39.552244+0000","last_clean":"2023-11-28T11:08:39.552244+0000","last_became_active":"2023-11-28T11:08:35.142697+0000","last_became_peered":"2023-11-28T11:08:35.142697+0000","last_unstale":"2023-11-28T11:08:39.552244+0000","last_undegraded":"2023-11-28T11:08:39.552244+0000","last_fullsized":"2023-11-28T11:08:39.552244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:24.895027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552639+0000","last_change":"2023-11-28T11:08:35.535083+0000","last_active":"2023-11-28T11:08:39.552639+0000","last_peered":"2023-11-28T11:08:39.552639+0000","last_clean":"2023-11-28T11:08:39.552639+0000","last_became_active":"2023-11-28T11:08:35.534800+0000","last_became_peered":"2023-11-28T11:08:35.534800+0000","last_unstale":"2023-11-28T11:08:39.552639+0000","last_undegraded":"2023-11-28T11:08:39.552639+0000","last_fullsized":"2023-11-28T11:08:39.552639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:25.900226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571669+0000","last_change":"2023-11-28T11:08:39.565208+0000","last_active":"2023-11-28T11:08:39.571669+0000","last_peered":"2023-11-28T11:08:39.571669+0000","last_clean":"2023-11-28T11:08:39.571669+0000","last_became_active":"2023-11-28T11:08:39.564299+0000","last_became_peered":"2023-11-28T11:08:39.564299+0000","last_unstale":"2023-11-28T11:08:39.571669+0000","last_undegraded":"2023-11-28T11:08:39.571669+0000","last_fullsized":"2023-11-28T11:08:39.571669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:48.486752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546343+0000","last_change":"2023-11-28T11:08:35.141725+0000","last_active":"2023-11-28T11:08:39.546343+0000","last_peered":"2023-11-28T11:08:39.546343+0000","last_clean":"2023-11-28T11:08:39.546343+0000","last_became_active":"2023-11-28T11:08:35.141445+0000","last_became_peered":"2023-11-28T11:08:35.141445+0000","last_unstale":"2023-11-28T11:08:39.546343+0000","last_undegraded":"2023-11-28T11:08:39.546343+0000","last_fullsized":"2023-11-28T11:08:39.546343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556057+0000","last_change":"2023-11-28T11:08:35.548819+0000","last_active":"2023-11-28T11:08:39.556057+0000","last_peered":"2023-11-28T11:08:39.556057+0000","last_clean":"2023-11-28T11:08:39.556057+0000","last_became_active":"2023-11-28T11:08:35.548403+0000","last_became_peered":"2023-11-28T11:08:35.548403+0000","last_unstale":"2023-11-28T11:08:39.556057+0000","last_undegraded":"2023-11-28T11:08:39.556057+0000","last_fullsized":"2023-11-28T11:08:39.556057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:45.068645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571605+0000","last_change":"2023-11-28T11:08:39.562941+0000","last_active":"2023-11-28T11:08:39.571605+0000","last_peered":"2023-11-28T11:08:39.571605+0000","last_clean":"2023-11-28T11:08:39.571605+0000","last_became_active":"2023-11-28T11:08:39.562555+0000","last_became_peered":"2023-11-28T11:08:39.562555+0000","last_unstale":"2023-11-28T11:08:39.571605+0000","last_undegraded":"2023-11-28T11:08:39.571605+0000","last_fullsized":"2023-11-28T11:08:39.571605+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:10.153438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544942+0000","last_change":"2023-11-28T11:08:35.142283+0000","last_active":"2023-11-28T11:08:39.544942+0000","last_peered":"2023-11-28T11:08:39.544942+0000","last_clean":"2023-11-28T11:08:39.544942+0000","last_became_active":"2023-11-28T11:08:35.141976+0000","last_became_peered":"2023-11-28T11:08:35.141976+0000","last_unstale":"2023-11-28T11:08:39.544942+0000","last_undegraded":"2023-11-28T11:08:39.544942+0000","last_fullsized":"2023-11-28T11:08:39.544942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556613+0000","last_change":"2023-11-28T11:08:35.540882+0000","last_active":"2023-11-28T11:08:39.556613+0000","last_peered":"2023-11-28T11:08:39.556613+0000","last_clean":"2023-11-28T11:08:39.556613+0000","last_became_active":"2023-11-28T11:08:35.540566+0000","last_became_peered":"2023-11-28T11:08:35.540566+0000","last_unstale":"2023-11-28T11:08:39.556613+0000","last_undegraded":"2023-11-28T11:08:39.556613+0000","last_fullsized":"2023-11-28T11:08:39.556613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:06.735331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570782+0000","last_change":"2023-11-28T11:08:39.566349+0000","last_active":"2023-11-28T11:08:39.570782+0000","last_peered":"2023-11-28T11:08:39.570782+0000","last_clean":"2023-11-28T11:08:39.570782+0000","last_became_active":"2023-11-28T11:08:39.565923+0000","last_became_peered":"2023-11-28T11:08:39.565923+0000","last_unstale":"2023-11-28T11:08:39.570782+0000","last_undegraded":"2023-11-28T11:08:39.570782+0000","last_fullsized":"2023-11-28T11:08:39.570782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551243+0000","last_change":"2023-11-28T11:08:35.146751+0000","last_active":"2023-11-28T11:08:39.551243+0000","last_peered":"2023-11-28T11:08:39.551243+0000","last_clean":"2023-11-28T11:08:39.551243+0000","last_became_active":"2023-11-28T11:08:35.146466+0000","last_became_peered":"2023-11-28T11:08:35.146466+0000","last_unstale":"2023-11-28T11:08:39.551243+0000","last_undegraded":"2023-11-28T11:08:39.551243+0000","last_fullsized":"2023-11-28T11:08:39.551243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556761+0000","last_change":"2023-11-28T11:08:35.540325+0000","last_active":"2023-11-28T11:08:39.556761+0000","last_peered":"2023-11-28T11:08:39.556761+0000","last_clean":"2023-11-28T11:08:39.556761+0000","last_became_active":"2023-11-28T11:08:35.540045+0000","last_became_peered":"2023-11-28T11:08:35.540045+0000","last_unstale":"2023-11-28T11:08:39.556761+0000","last_undegraded":"2023-11-28T11:08:39.556761+0000","last_fullsized":"2023-11-28T11:08:39.556761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:26.229967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566338+0000","last_change":"2023-11-28T11:08:39.562874+0000","last_active":"2023-11-28T11:08:39.566338+0000","last_peered":"2023-11-28T11:08:39.566338+0000","last_clean":"2023-11-28T11:08:39.566338+0000","last_became_active":"2023-11-28T11:08:39.562484+0000","last_became_peered":"2023-11-28T11:08:39.562484+0000","last_unstale":"2023-11-28T11:08:39.566338+0000","last_undegraded":"2023-11-28T11:08:39.566338+0000","last_fullsized":"2023-11-28T11:08:39.566338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:43.764613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552660+0000","last_change":"2023-11-28T11:08:35.139637+0000","last_active":"2023-11-28T11:08:39.552660+0000","last_peered":"2023-11-28T11:08:39.552660+0000","last_clean":"2023-11-28T11:08:39.552660+0000","last_became_active":"2023-11-28T11:08:35.138268+0000","last_became_peered":"2023-11-28T11:08:35.138268+0000","last_unstale":"2023-11-28T11:08:39.552660+0000","last_undegraded":"2023-11-28T11:08:39.552660+0000","last_fullsized":"2023-11-28T11:08:39.552660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556539+0000","last_change":"2023-11-28T11:08:35.543314+0000","last_active":"2023-11-28T11:08:39.556539+0000","last_peered":"2023-11-28T11:08:39.556539+0000","last_clean":"2023-11-28T11:08:39.556539+0000","last_became_active":"2023-11-28T11:08:35.539838+0000","last_became_peered":"2023-11-28T11:08:35.539838+0000","last_unstale":"2023-11-28T11:08:39.556539+0000","last_undegraded":"2023-11-28T11:08:39.556539+0000","last_fullsized":"2023-11-28T11:08:39.556539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:53.884226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568846+0000","last_change":"2023-11-28T11:08:39.565918+0000","last_active":"2023-11-28T11:08:39.568846+0000","last_peered":"2023-11-28T11:08:39.568846+0000","last_clean":"2023-11-28T11:08:39.568846+0000","last_became_active":"2023-11-28T11:08:39.562911+0000","last_became_peered":"2023-11-28T11:08:39.562911+0000","last_unstale":"2023-11-28T11:08:39.568846+0000","last_undegraded":"2023-11-28T11:08:39.568846+0000","last_fullsized":"2023-11-28T11:08:39.568846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:11:09.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546326+0000","last_change":"2023-11-28T11:08:35.143397+0000","last_active":"2023-11-28T11:08:39.546326+0000","last_peered":"2023-11-28T11:08:39.546326+0000","last_clean":"2023-11-28T11:08:39.546326+0000","last_became_active":"2023-11-28T11:08:35.142466+0000","last_became_peered":"2023-11-28T11:08:35.142466+0000","last_unstale":"2023-11-28T11:08:39.546326+0000","last_undegraded":"2023-11-28T11:08:39.546326+0000","last_fullsized":"2023-11-28T11:08:39.546326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556168+0000","last_change":"2023-11-28T11:08:35.534626+0000","last_active":"2023-11-28T11:08:39.556168+0000","last_peered":"2023-11-28T11:08:39.556168+0000","last_clean":"2023-11-28T11:08:39.556168+0000","last_became_active":"2023-11-28T11:08:35.534379+0000","last_became_peered":"2023-11-28T11:08:35.534379+0000","last_unstale":"2023-11-28T11:08:39.556168+0000","last_undegraded":"2023-11-28T11:08:39.556168+0000","last_fullsized":"2023-11-28T11:08:39.556168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:21.106849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.573390+0000","last_change":"2023-11-28T11:08:39.569661+0000","last_active":"2023-11-28T11:08:39.573390+0000","last_peered":"2023-11-28T11:08:39.573390+0000","last_clean":"2023-11-28T11:08:39.573390+0000","last_became_active":"2023-11-28T11:08:39.569175+0000","last_became_peered":"2023-11-28T11:08:39.569175+0000","last_unstale":"2023-11-28T11:08:39.573390+0000","last_undegraded":"2023-11-28T11:08:39.573390+0000","last_fullsized":"2023-11-28T11:08:39.573390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:59.446120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544982+0000","last_change":"2023-11-28T11:08:35.143399+0000","last_active":"2023-11-28T11:08:39.544982+0000","last_peered":"2023-11-28T11:08:39.544982+0000","last_clean":"2023-11-28T11:08:39.544982+0000","last_became_active":"2023-11-28T11:08:35.143102+0000","last_became_peered":"2023-11-28T11:08:35.143102+0000","last_unstale":"2023-11-28T11:08:39.544982+0000","last_undegraded":"2023-11-28T11:08:39.544982+0000","last_fullsized":"2023-11-28T11:08:39.544982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556842+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:08:39.556842+0000","last_peered":"2023-11-28T11:08:39.556842+0000","last_clean":"2023-11-28T11:08:39.556842+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:08:39.556842+0000","last_undegraded":"2023-11-28T11:08:39.556842+0000","last_fullsized":"2023-11-28T11:08:39.556842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545915+0000","last_change":"2023-11-28T11:08:35.541344+0000","last_active":"2023-11-28T11:08:39.545915+0000","last_peered":"2023-11-28T11:08:39.545915+0000","last_clean":"2023-11-28T11:08:39.545915+0000","last_became_active":"2023-11-28T11:08:35.541086+0000","last_became_peered":"2023-11-28T11:08:35.541086+0000","last_unstale":"2023-11-28T11:08:39.545915+0000","last_undegraded":"2023-11-28T11:08:39.545915+0000","last_fullsized":"2023-11-28T11:08:39.545915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.329131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.572142+0000","last_change":"2023-11-28T11:08:39.567333+0000","last_active":"2023-11-28T11:08:39.572142+0000","last_peered":"2023-11-28T11:08:39.572142+0000","last_clean":"2023-11-28T11:08:39.572142+0000","last_became_active":"2023-11-28T11:08:39.566952+0000","last_became_peered":"2023-11-28T11:08:39.566952+0000","last_unstale":"2023-11-28T11:08:39.572142+0000","last_undegraded":"2023-11-28T11:08:39.572142+0000","last_fullsized":"2023-11-28T11:08:39.572142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:27.385978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.545875+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:08:39.545875+0000","last_peered":"2023-11-28T11:08:39.545875+0000","last_clean":"2023-11-28T11:08:39.545875+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:08:39.545875+0000","last_undegraded":"2023-11-28T11:08:39.545875+0000","last_fullsized":"2023-11-28T11:08:39.545875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:40.959549+0000","last_change":"2023-11-28T11:08:35.138175+0000","last_active":"2023-11-28T11:08:40.959549+0000","last_peered":"2023-11-28T11:08:40.959549+0000","last_clean":"2023-11-28T11:08:40.959549+0000","last_became_active":"2023-11-28T11:08:35.137577+0000","last_became_peered":"2023-11-28T11:08:35.137577+0000","last_unstale":"2023-11-28T11:08:40.959549+0000","last_undegraded":"2023-11-28T11:08:40.959549+0000","last_fullsized":"2023-11-28T11:08:40.959549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.560017+0000","last_change":"2023-11-28T11:08:35.540521+0000","last_active":"2023-11-28T11:08:39.560017+0000","last_peered":"2023-11-28T11:08:39.560017+0000","last_clean":"2023-11-28T11:08:39.560017+0000","last_became_active":"2023-11-28T11:08:35.538206+0000","last_became_peered":"2023-11-28T11:08:35.538206+0000","last_unstale":"2023-11-28T11:08:39.560017+0000","last_undegraded":"2023-11-28T11:08:39.560017+0000","last_fullsized":"2023-11-28T11:08:39.560017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568578+0000","last_change":"2023-11-28T11:08:39.565755+0000","last_active":"2023-11-28T11:08:39.568578+0000","last_peered":"2023-11-28T11:08:39.568578+0000","last_clean":"2023-11-28T11:08:39.568578+0000","last_became_active":"2023-11-28T11:08:39.565093+0000","last_became_peered":"2023-11-28T11:08:39.565093+0000","last_unstale":"2023-11-28T11:08:39.568578+0000","last_undegraded":"2023-11-28T11:08:39.568578+0000","last_fullsized":"2023-11-28T11:08:39.568578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:40.034910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555643+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:08:39.555643+0000","last_peered":"2023-11-28T11:08:39.555643+0000","last_clean":"2023-11-28T11:08:39.555643+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:08:39.555643+0000","last_undegraded":"2023-11-28T11:08:39.555643+0000","last_fullsized":"2023-11-28T11:08:39.555643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550860+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:08:39.550860+0000","last_peered":"2023-11-28T11:08:39.550860+0000","last_clean":"2023-11-28T11:08:39.550860+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:08:39.550860+0000","last_undegraded":"2023-11-28T11:08:39.550860+0000","last_fullsized":"2023-11-28T11:08:39.550860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.549175+0000","last_change":"2023-11-28T11:08:35.146578+0000","last_active":"2023-11-28T11:08:39.549175+0000","last_peered":"2023-11-28T11:08:39.549175+0000","last_clean":"2023-11-28T11:08:39.549175+0000","last_became_active":"2023-11-28T11:08:35.146300+0000","last_became_peered":"2023-11-28T11:08:35.146300+0000","last_unstale":"2023-11-28T11:08:39.549175+0000","last_undegraded":"2023-11-28T11:08:39.549175+0000","last_fullsized":"2023-11-28T11:08:39.549175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:20.573471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552070+0000","last_change":"2023-11-28T11:08:35.545787+0000","last_active":"2023-11-28T11:08:39.552070+0000","last_peered":"2023-11-28T11:08:39.552070+0000","last_clean":"2023-11-28T11:08:39.552070+0000","last_became_active":"2023-11-28T11:08:35.544359+0000","last_became_peered":"2023-11-28T11:08:35.544359+0000","last_unstale":"2023-11-28T11:08:39.552070+0000","last_undegraded":"2023-11-28T11:08:39.552070+0000","last_fullsized":"2023-11-28T11:08:39.552070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.561183+0000","last_change":"2023-11-28T11:08:39.558516+0000","last_active":"2023-11-28T11:08:39.561183+0000","last_peered":"2023-11-28T11:08:39.561183+0000","last_clean":"2023-11-28T11:08:39.561183+0000","last_became_active":"2023-11-28T11:08:39.558117+0000","last_became_peered":"2023-11-28T11:08:39.558117+0000","last_unstale":"2023-11-28T11:08:39.561183+0000","last_undegraded":"2023-11-28T11:08:39.561183+0000","last_fullsized":"2023-11-28T11:08:39.561183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551550+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:08:39.551550+0000","last_peered":"2023-11-28T11:08:39.551550+0000","last_clean":"2023-11-28T11:08:39.551550+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:08:39.551550+0000","last_undegraded":"2023-11-28T11:08:39.551550+0000","last_fullsized":"2023-11-28T11:08:39.551550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.554459+0000","last_change":"2023-11-28T11:08:35.134990+0000","last_active":"2023-11-28T11:08:39.554459+0000","last_peered":"2023-11-28T11:08:39.554459+0000","last_clean":"2023-11-28T11:08:39.554459+0000","last_became_active":"2023-11-28T11:08:35.134728+0000","last_became_peered":"2023-11-28T11:08:35.134728+0000","last_unstale":"2023-11-28T11:08:39.554459+0000","last_undegraded":"2023-11-28T11:08:39.554459+0000","last_fullsized":"2023-11-28T11:08:39.554459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556117+0000","last_change":"2023-11-28T11:08:35.540730+0000","last_active":"2023-11-28T11:08:39.556117+0000","last_peered":"2023-11-28T11:08:39.556117+0000","last_clean":"2023-11-28T11:08:39.556117+0000","last_became_active":"2023-11-28T11:08:35.540312+0000","last_became_peered":"2023-11-28T11:08:35.540312+0000","last_unstale":"2023-11-28T11:08:39.556117+0000","last_undegraded":"2023-11-28T11:08:39.556117+0000","last_fullsized":"2023-11-28T11:08:39.556117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.140756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568532+0000","last_change":"2023-11-28T11:08:39.561818+0000","last_active":"2023-11-28T11:08:39.568532+0000","last_peered":"2023-11-28T11:08:39.568532+0000","last_clean":"2023-11-28T11:08:39.568532+0000","last_became_active":"2023-11-28T11:08:39.561292+0000","last_became_peered":"2023-11-28T11:08:39.561292+0000","last_unstale":"2023-11-28T11:08:39.568532+0000","last_undegraded":"2023-11-28T11:08:39.568532+0000","last_fullsized":"2023-11-28T11:08:39.568532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550685+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:08:39.550685+0000","last_peered":"2023-11-28T11:08:39.550685+0000","last_clean":"2023-11-28T11:08:39.550685+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:08:39.550685+0000","last_undegraded":"2023-11-28T11:08:39.550685+0000","last_fullsized":"2023-11-28T11:08:39.550685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546311+0000","last_change":"2023-11-28T11:08:35.140783+0000","last_active":"2023-11-28T11:08:39.546311+0000","last_peered":"2023-11-28T11:08:39.546311+0000","last_clean":"2023-11-28T11:08:39.546311+0000","last_became_active":"2023-11-28T11:08:35.140512+0000","last_became_peered":"2023-11-28T11:08:35.140512+0000","last_unstale":"2023-11-28T11:08:39.546311+0000","last_undegraded":"2023-11-28T11:08:39.546311+0000","last_fullsized":"2023-11-28T11:08:39.546311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551352+0000","last_change":"2023-11-28T11:08:35.547824+0000","last_active":"2023-11-28T11:08:39.551352+0000","last_peered":"2023-11-28T11:08:39.551352+0000","last_clean":"2023-11-28T11:08:39.551352+0000","last_became_active":"2023-11-28T11:08:35.547517+0000","last_became_peered":"2023-11-28T11:08:35.547517+0000","last_unstale":"2023-11-28T11:08:39.551352+0000","last_undegraded":"2023-11-28T11:08:39.551352+0000","last_fullsized":"2023-11-28T11:08:39.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:55.715375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570503+0000","last_change":"2023-11-28T11:08:39.564906+0000","last_active":"2023-11-28T11:08:39.570503+0000","last_peered":"2023-11-28T11:08:39.570503+0000","last_clean":"2023-11-28T11:08:39.570503+0000","last_became_active":"2023-11-28T11:08:39.564258+0000","last_became_peered":"2023-11-28T11:08:39.564258+0000","last_unstale":"2023-11-28T11:08:39.570503+0000","last_undegraded":"2023-11-28T11:08:39.570503+0000","last_fullsized":"2023-11-28T11:08:39.570503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:43.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546086+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:08:39.546086+0000","last_peered":"2023-11-28T11:08:39.546086+0000","last_clean":"2023-11-28T11:08:39.546086+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:08:39.546086+0000","last_undegraded":"2023-11-28T11:08:39.546086+0000","last_fullsized":"2023-11-28T11:08:39.546086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556375+0000","last_change":"2023-11-28T11:08:35.144005+0000","last_active":"2023-11-28T11:08:39.556375+0000","last_peered":"2023-11-28T11:08:39.556375+0000","last_clean":"2023-11-28T11:08:39.556375+0000","last_became_active":"2023-11-28T11:08:35.141190+0000","last_became_peered":"2023-11-28T11:08:35.141190+0000","last_unstale":"2023-11-28T11:08:39.556375+0000","last_undegraded":"2023-11-28T11:08:39.556375+0000","last_fullsized":"2023-11-28T11:08:39.556375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555324+0000","last_change":"2023-11-28T11:08:35.539604+0000","last_active":"2023-11-28T11:08:39.555324+0000","last_peered":"2023-11-28T11:08:39.555324+0000","last_clean":"2023-11-28T11:08:39.555324+0000","last_became_active":"2023-11-28T11:08:35.539319+0000","last_became_peered":"2023-11-28T11:08:35.539319+0000","last_unstale":"2023-11-28T11:08:39.555324+0000","last_undegraded":"2023-11-28T11:08:39.555324+0000","last_fullsized":"2023-11-28T11:08:39.555324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:00.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.562105+0000","last_change":"2023-11-28T11:08:39.559157+0000","last_active":"2023-11-28T11:08:39.562105+0000","last_peered":"2023-11-28T11:08:39.562105+0000","last_clean":"2023-11-28T11:08:39.562105+0000","last_became_active":"2023-11-28T11:08:39.558790+0000","last_became_peered":"2023-11-28T11:08:39.558790+0000","last_unstale":"2023-11-28T11:08:39.562105+0000","last_undegraded":"2023-11-28T11:08:39.562105+0000","last_fullsized":"2023-11-28T11:08:39.562105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:18.469803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.550458+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:08:39.550458+0000","last_peered":"2023-11-28T11:08:39.550458+0000","last_clean":"2023-11-28T11:08:39.550458+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:08:39.550458+0000","last_undegraded":"2023-11-28T11:08:39.550458+0000","last_fullsized":"2023-11-28T11:08:39.550458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555493+0000","last_change":"2023-11-28T11:08:35.136293+0000","last_active":"2023-11-28T11:08:39.555493+0000","last_peered":"2023-11-28T11:08:39.555493+0000","last_clean":"2023-11-28T11:08:39.555493+0000","last_became_active":"2023-11-28T11:08:35.136011+0000","last_became_peered":"2023-11-28T11:08:35.136011+0000","last_unstale":"2023-11-28T11:08:39.555493+0000","last_undegraded":"2023-11-28T11:08:39.555493+0000","last_fullsized":"2023-11-28T11:08:39.555493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:10.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556312+0000","last_change":"2023-11-28T11:08:35.548876+0000","last_active":"2023-11-28T11:08:39.556312+0000","last_peered":"2023-11-28T11:08:39.556312+0000","last_clean":"2023-11-28T11:08:39.556312+0000","last_became_active":"2023-11-28T11:08:35.548606+0000","last_became_peered":"2023-11-28T11:08:35.548606+0000","last_unstale":"2023-11-28T11:08:39.556312+0000","last_undegraded":"2023-11-28T11:08:39.556312+0000","last_fullsized":"2023-11-28T11:08:39.556312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570495+0000","last_change":"2023-11-28T11:08:39.564356+0000","last_active":"2023-11-28T11:08:39.570495+0000","last_peered":"2023-11-28T11:08:39.570495+0000","last_clean":"2023-11-28T11:08:39.570495+0000","last_became_active":"2023-11-28T11:08:39.563998+0000","last_became_peered":"2023-11-28T11:08:39.563998+0000","last_unstale":"2023-11-28T11:08:39.570495+0000","last_undegraded":"2023-11-28T11:08:39.570495+0000","last_fullsized":"2023-11-28T11:08:39.570495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.553351+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:08:39.553351+0000","last_peered":"2023-11-28T11:08:39.553351+0000","last_clean":"2023-11-28T11:08:39.553351+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:08:39.553351+0000","last_undegraded":"2023-11-28T11:08:39.553351+0000","last_fullsized":"2023-11-28T11:08:39.553351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546501+0000","last_change":"2023-11-28T11:08:35.141912+0000","last_active":"2023-11-28T11:08:39.546501+0000","last_peered":"2023-11-28T11:08:39.546501+0000","last_clean":"2023-11-28T11:08:39.546501+0000","last_became_active":"2023-11-28T11:08:35.141586+0000","last_became_peered":"2023-11-28T11:08:35.141586+0000","last_unstale":"2023-11-28T11:08:39.546501+0000","last_undegraded":"2023-11-28T11:08:39.546501+0000","last_fullsized":"2023-11-28T11:08:39.546501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:51.806168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552908+0000","last_change":"2023-11-28T11:08:35.542267+0000","last_active":"2023-11-28T11:08:39.552908+0000","last_peered":"2023-11-28T11:08:39.552908+0000","last_clean":"2023-11-28T11:08:39.552908+0000","last_became_active":"2023-11-28T11:08:35.541323+0000","last_became_peered":"2023-11-28T11:08:35.541323+0000","last_unstale":"2023-11-28T11:08:39.552908+0000","last_undegraded":"2023-11-28T11:08:39.552908+0000","last_fullsized":"2023-11-28T11:08:39.552908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:43.766043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570775+0000","last_change":"2023-11-28T11:08:39.566246+0000","last_active":"2023-11-28T11:08:39.570775+0000","last_peered":"2023-11-28T11:08:39.570775+0000","last_clean":"2023-11-28T11:08:39.570775+0000","last_became_active":"2023-11-28T11:08:39.565895+0000","last_became_peered":"2023-11-28T11:08:39.565895+0000","last_unstale":"2023-11-28T11:08:39.570775+0000","last_undegraded":"2023-11-28T11:08:39.570775+0000","last_fullsized":"2023-11-28T11:08:39.570775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:52.787854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556229+0000","last_change":"2023-11-28T11:08:35.146576+0000","last_active":"2023-11-28T11:08:39.556229+0000","last_peered":"2023-11-28T11:08:39.556229+0000","last_clean":"2023-11-28T11:08:39.556229+0000","last_became_active":"2023-11-28T11:08:35.146283+0000","last_became_peered":"2023-11-28T11:08:35.146283+0000","last_unstale":"2023-11-28T11:08:39.556229+0000","last_undegraded":"2023-11-28T11:08:39.556229+0000","last_fullsized":"2023-11-28T11:08:39.556229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.542850+0000","last_change":"2023-11-28T11:08:35.536348+0000","last_active":"2023-11-28T11:08:39.542850+0000","last_peered":"2023-11-28T11:08:39.542850+0000","last_clean":"2023-11-28T11:08:39.542850+0000","last_became_active":"2023-11-28T11:08:35.536095+0000","last_became_peered":"2023-11-28T11:08:35.536095+0000","last_unstale":"2023-11-28T11:08:39.542850+0000","last_undegraded":"2023-11-28T11:08:39.542850+0000","last_fullsized":"2023-11-28T11:08:39.542850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.991268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.570654+0000","last_change":"2023-11-28T11:08:39.567740+0000","last_active":"2023-11-28T11:08:39.570654+0000","last_peered":"2023-11-28T11:08:39.570654+0000","last_clean":"2023-11-28T11:08:39.570654+0000","last_became_active":"2023-11-28T11:08:39.567279+0000","last_became_peered":"2023-11-28T11:08:39.567279+0000","last_unstale":"2023-11-28T11:08:39.570654+0000","last_undegraded":"2023-11-28T11:08:39.570654+0000","last_fullsized":"2023-11-28T11:08:39.570654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546145+0000","last_change":"2023-11-28T11:08:35.141111+0000","last_active":"2023-11-28T11:08:39.546145+0000","last_peered":"2023-11-28T11:08:39.546145+0000","last_clean":"2023-11-28T11:08:39.546145+0000","last_became_active":"2023-11-28T11:08:35.140804+0000","last_became_peered":"2023-11-28T11:08:35.140804+0000","last_unstale":"2023-11-28T11:08:39.546145+0000","last_undegraded":"2023-11-28T11:08:39.546145+0000","last_fullsized":"2023-11-28T11:08:39.546145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:36.986069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556281+0000","last_change":"2023-11-28T11:08:35.547330+0000","last_active":"2023-11-28T11:08:39.556281+0000","last_peered":"2023-11-28T11:08:39.556281+0000","last_clean":"2023-11-28T11:08:39.556281+0000","last_became_active":"2023-11-28T11:08:35.546970+0000","last_became_peered":"2023-11-28T11:08:35.546970+0000","last_unstale":"2023-11-28T11:08:39.556281+0000","last_undegraded":"2023-11-28T11:08:39.556281+0000","last_fullsized":"2023-11-28T11:08:39.556281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:28.259978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.568054+0000","last_change":"2023-11-28T11:08:39.565657+0000","last_active":"2023-11-28T11:08:39.568054+0000","last_peered":"2023-11-28T11:08:39.568054+0000","last_clean":"2023-11-28T11:08:39.568054+0000","last_became_active":"2023-11-28T11:08:39.565004+0000","last_became_peered":"2023-11-28T11:08:39.565004+0000","last_unstale":"2023-11-28T11:08:39.568054+0000","last_undegraded":"2023-11-28T11:08:39.568054+0000","last_fullsized":"2023-11-28T11:08:39.568054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.546414+0000","last_change":"2023-11-28T11:08:35.146886+0000","last_active":"2023-11-28T11:08:39.546414+0000","last_peered":"2023-11-28T11:08:39.546414+0000","last_clean":"2023-11-28T11:08:39.546414+0000","last_became_active":"2023-11-28T11:08:35.146600+0000","last_became_peered":"2023-11-28T11:08:35.146600+0000","last_unstale":"2023-11-28T11:08:39.546414+0000","last_undegraded":"2023-11-28T11:08:39.546414+0000","last_fullsized":"2023-11-28T11:08:39.546414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:34.287972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556350+0000","last_change":"2023-11-28T11:08:35.538328+0000","last_active":"2023-11-28T11:08:39.556350+0000","last_peered":"2023-11-28T11:08:39.556350+0000","last_clean":"2023-11-28T11:08:39.556350+0000","last_became_active":"2023-11-28T11:08:35.537863+0000","last_became_peered":"2023-11-28T11:08:35.537863+0000","last_unstale":"2023-11-28T11:08:39.556350+0000","last_undegraded":"2023-11-28T11:08:39.556350+0000","last_fullsized":"2023-11-28T11:08:39.556350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:15.051695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.566546+0000","last_change":"2023-11-28T11:08:39.562424+0000","last_active":"2023-11-28T11:08:39.566546+0000","last_peered":"2023-11-28T11:08:39.566546+0000","last_clean":"2023-11-28T11:08:39.566546+0000","last_became_active":"2023-11-28T11:08:39.562065+0000","last_became_peered":"2023-11-28T11:08:39.562065+0000","last_unstale":"2023-11-28T11:08:39.566546+0000","last_undegraded":"2023-11-28T11:08:39.566546+0000","last_fullsized":"2023-11-28T11:08:39.566546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:39.095254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544783+0000","last_change":"2023-11-28T11:08:35.138855+0000","last_active":"2023-11-28T11:08:39.544783+0000","last_peered":"2023-11-28T11:08:39.544783+0000","last_clean":"2023-11-28T11:08:39.544783+0000","last_became_active":"2023-11-28T11:08:35.138526+0000","last_became_peered":"2023-11-28T11:08:35.138526+0000","last_unstale":"2023-11-28T11:08:39.544783+0000","last_undegraded":"2023-11-28T11:08:39.544783+0000","last_fullsized":"2023-11-28T11:08:39.544783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:55.811843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544513+0000","last_change":"2023-11-28T11:08:35.539629+0000","last_active":"2023-11-28T11:08:39.544513+0000","last_peered":"2023-11-28T11:08:39.544513+0000","last_clean":"2023-11-28T11:08:39.544513+0000","last_became_active":"2023-11-28T11:08:35.534961+0000","last_became_peered":"2023-11-28T11:08:35.534961+0000","last_unstale":"2023-11-28T11:08:39.544513+0000","last_undegraded":"2023-11-28T11:08:39.544513+0000","last_fullsized":"2023-11-28T11:08:39.544513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:22.705182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.571346+0000","last_change":"2023-11-28T11:08:39.570212+0000","last_active":"2023-11-28T11:08:39.571346+0000","last_peered":"2023-11-28T11:08:39.571346+0000","last_clean":"2023-11-28T11:08:39.571346+0000","last_became_active":"2023-11-28T11:08:39.569900+0000","last_became_peered":"2023-11-28T11:08:39.569900+0000","last_unstale":"2023-11-28T11:08:39.571346+0000","last_undegraded":"2023-11-28T11:08:39.571346+0000","last_fullsized":"2023-11-28T11:08:39.571346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:38.534496+0000","last_clean_scrub_stamp":"2023-11-28T11:08:38.534496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:19.435944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552849+0000","last_change":"2023-11-28T11:08:35.139415+0000","last_active":"2023-11-28T11:08:39.552849+0000","last_peered":"2023-11-28T11:08:39.552849+0000","last_clean":"2023-11-28T11:08:39.552849+0000","last_became_active":"2023-11-28T11:08:35.139152+0000","last_became_peered":"2023-11-28T11:08:35.139152+0000","last_unstale":"2023-11-28T11:08:39.552849+0000","last_undegraded":"2023-11-28T11:08:39.552849+0000","last_fullsized":"2023-11-28T11:08:39.552849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:34.111190+0000","last_clean_scrub_stamp":"2023-11-28T11:08:34.111190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:40.421766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.552466+0000","last_change":"2023-11-28T11:08:35.544319+0000","last_active":"2023-11-28T11:08:39.552466+0000","last_peered":"2023-11-28T11:08:39.552466+0000","last_clean":"2023-11-28T11:08:39.552466+0000","last_became_active":"2023-11-28T11:08:35.544032+0000","last_became_peered":"2023-11-28T11:08:35.544032+0000","last_unstale":"2023-11-28T11:08:39.552466+0000","last_undegraded":"2023-11-28T11:08:39.552466+0000","last_fullsized":"2023-11-28T11:08:39.552466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:36.189248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.551454+0000","last_change":"2023-11-28T11:08:35.534208+0000","last_active":"2023-11-28T11:08:39.551454+0000","last_peered":"2023-11-28T11:08:39.551454+0000","last_clean":"2023-11-28T11:08:39.551454+0000","last_became_active":"2023-11-28T11:08:35.533929+0000","last_became_peered":"2023-11-28T11:08:35.533929+0000","last_unstale":"2023-11-28T11:08:39.551454+0000","last_undegraded":"2023-11-28T11:08:39.551454+0000","last_fullsized":"2023-11-28T11:08:39.551454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:44.139630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544658+0000","last_change":"2023-11-28T11:08:35.540664+0000","last_active":"2023-11-28T11:08:39.544658+0000","last_peered":"2023-11-28T11:08:39.544658+0000","last_clean":"2023-11-28T11:08:39.544658+0000","last_became_active":"2023-11-28T11:08:35.540243+0000","last_became_peered":"2023-11-28T11:08:35.540243+0000","last_unstale":"2023-11-28T11:08:39.544658+0000","last_undegraded":"2023-11-28T11:08:39.544658+0000","last_fullsized":"2023-11-28T11:08:39.544658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:24.130792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.556410+0000","last_change":"2023-11-28T11:08:35.544372+0000","last_active":"2023-11-28T11:08:39.556410+0000","last_peered":"2023-11-28T11:08:39.556410+0000","last_clean":"2023-11-28T11:08:39.556410+0000","last_became_active":"2023-11-28T11:08:35.543867+0000","last_became_peered":"2023-11-28T11:08:35.543867+0000","last_unstale":"2023-11-28T11:08:39.556410+0000","last_undegraded":"2023-11-28T11:08:39.556410+0000","last_fullsized":"2023-11-28T11:08:39.556410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:30.327766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.544832+0000","last_change":"2023-11-28T11:08:35.543075+0000","last_active":"2023-11-28T11:08:39.544832+0000","last_peered":"2023-11-28T11:08:39.544832+0000","last_clean":"2023-11-28T11:08:39.544832+0000","last_became_active":"2023-11-28T11:08:35.542645+0000","last_became_peered":"2023-11-28T11:08:35.542645+0000","last_unstale":"2023-11-28T11:08:39.544832+0000","last_undegraded":"2023-11-28T11:08:39.544832+0000","last_fullsized":"2023-11-28T11:08:39.544832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:08.741004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:39.555411+0000","last_change":"2023-11-28T11:08:35.546225+0000","last_active":"2023-11-28T11:08:39.555411+0000","last_peered":"2023-11-28T11:08:39.555411+0000","last_clean":"2023-11-28T11:08:39.555411+0000","last_became_active":"2023-11-28T11:08:35.545970+0000","last_became_peered":"2023-11-28T11:08:35.545970+0000","last_unstale":"2023-11-28T11:08:39.555411+0000","last_undegraded":"2023-11-28T11:08:39.555411+0000","last_fullsized":"2023-11-28T11:08:39.555411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:35.116389+0000","last_clean_scrub_stamp":"2023-11-28T11:08:35.116389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:07.219186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9248,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362592,"statfs":{"total":96636764160,"available":96627294208,"internally_reserved":0,"allocated":876544,"data_stored":699060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8060,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363780,"statfs":{"total":96636764160,"available":96628510720,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8128,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363712,"statfs":{"total":96636764160,"available":96628441088,"internally_reserved":0,"allocated":253952,"data_stored":103935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8064,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363776,"statfs":{"total":96636764160,"available":96628506624,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8132,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363708,"statfs":{"total":96636764160,"available":96628436992,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8212,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363628,"statfs":{"total":96636764160,"available":96628355072,"internally_reserved":0,"allocated":274432,"data_stored":108532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":105366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:57.901 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:08:57.901 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T11:08:57.901 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T11:08:57.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:08:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.052+0000 7f96e3aa3700 1 Processor -- start 2023-11-28T11:08:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e3aa3700 1 -- start start 2023-11-28T11:08:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e3aa3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96dc159830 0x7f96dc0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e3aa3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f96dc0b18a0 0x7f96dc0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e3aa3700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0b2220 con 0x7f96dc06c5f0 2023-11-28T11:08:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e3aa3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc15d480 con 0x7f96dc159830 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e3aa3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc06ca60 con 0x7f96dc0b18a0 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.053+0000 7f96e183f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96dc159830 0x7f96dc0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96dc159830 0x7f96dc0cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49132/0 (socket says 172.21.15.83:49132) 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 -- 172.21.15.83:0/4283290430 learned_addr learned my addr 172.21.15.83:0/4283290430 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e2040700 1 --2- 172.21.15.83:0/4283290430 >> v2:172.21.15.90:3300/0 conn(0x7f96dc0b18a0 0x7f96dc0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 -- 172.21.15.83:0/4283290430 >> v2:172.21.15.90:3300/0 conn(0x7f96dc0b18a0 msgr2=0x7f96dc0b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 --2- 172.21.15.83:0/4283290430 >> v2:172.21.15.90:3300/0 conn(0x7f96dc0b18a0 0x7f96dc0b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 -- 172.21.15.83:0/4283290430 >> v1:172.21.15.90:6789/0 conn(0x7f96dc06c5f0 legacy=0x7f96dc15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 -- 172.21.15.83:0/4283290430 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc0cf790 con 0x7f96dc159830 2023-11-28T11:08:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e2040700 1 --2- 172.21.15.83:0/4283290430 >> v2:172.21.15.90:3300/0 conn(0x7f96dc0b18a0 0x7f96dc0b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e183f700 1 --2- 172.21.15.83:0/4283290430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc159830 0x7f96dc0cf0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0b2790 tx=0x7f96cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fe60d7451aebc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e083d700 1 -- 172.21.15.83:0/4283290430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc004230 con 0x7f96dc159830 2023-11-28T11:08:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e083d700 1 -- 172.21.15.83:0/4283290430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc004390 con 0x7f96dc159830 2023-11-28T11:08:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.054+0000 7f96e083d700 1 -- 172.21.15.83:0/4283290430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0036a0 con 0x7f96dc159830 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 -- 172.21.15.83:0/4283290430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc159830 msgr2=0x7f96dc0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/4283290430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc159830 0x7f96dc0cf0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0b2790 tx=0x7f96cc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 -- 172.21.15.83:0/4283290430 shutdown_connections 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/4283290430 >> v2:172.21.15.90:3300/0 conn(0x7f96dc0b18a0 0x7f96dc0b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/4283290430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc159830 0x7f96dc0cf0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 -- 172.21.15.83:0/4283290430 >> 172.21.15.83:0/4283290430 conn(0x7f96dc06e9a0 msgr2=0x7f96dc0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 -- 172.21.15.83:0/4283290430 shutdown_connections 2023-11-28T11:08:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.055+0000 7f96e3aa3700 1 -- 172.21.15.83:0/4283290430 wait complete. 2023-11-28T11:08:58.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.056+0000 7f96e3aa3700 1 Processor -- start 2023-11-28T11:08:58.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.056+0000 7f96e3aa3700 1 -- start start 2023-11-28T11:08:58.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 0x7f96dc1e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96dc0b18a0 0x7f96dc1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f96dc159830 0x7f96dc0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0ac7b0 con 0x7f96dc06c5f0 2023-11-28T11:08:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc155710 con 0x7f96dc0b18a0 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1e4cc0 con 0x7f96dc159830 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 0x7f96dc1e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 0x7f96dc1e45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49138/0 (socket says 172.21.15.83:49138) 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 -- 172.21.15.83:0/698720882 learned_addr learned my addr 172.21.15.83:0/698720882 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e103e700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96dc0b18a0 0x7f96dc1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e2040700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f96dc159830 0x7f96dc0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 -- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f96dc159830 msgr2=0x7f96dc0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f96dc159830 0x7f96dc0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 -- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96dc0b18a0 msgr2=0x7f96dc1546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96dc0b18a0 0x7f96dc1546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 -- 172.21.15.83:0/698720882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009550 con 0x7f96dc06c5f0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e183f700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 0x7f96dc1e45e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009fd0 tx=0x7f96cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc003e20 con 0x7f96dc06c5f0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc011630 con 0x7f96dc06c5f0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc15d690 con 0x7f96dc06c5f0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0119e0 con 0x7f96dc06c5f0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.057+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc0cfa70 con 0x7f96dc06c5f0 2023-11-28T11:08:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.058+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc15d970 con 0x7f96dc06c5f0 2023-11-28T11:08:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.059+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f96cc017020 con 0x7f96dc06c5f0 2023-11-28T11:08:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.059+0000 7f96d27fc700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96c00682f0 0x7f96c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.059+0000 7f96e103e700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96c00682f0 0x7f96c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.059+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f96cc087400 con 0x7f96dc06c5f0 2023-11-28T11:08:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.060+0000 7f96e103e700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96c00682f0 0x7f96c006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f96dc158df0 tx=0x7f96d8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.064+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f96cc053970 con 0x7f96dc06c5f0 2023-11-28T11:08:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.288+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f96dc074320 con 0x7f96dc06c5f0 2023-11-28T11:08:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.288+0000 7f96d27fc700 1 -- 172.21.15.83:0/698720882 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f96cc056f90 con 0x7f96dc06c5f0 2023-11-28T11:08:58.290 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:58.290 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T11:08:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96c00682f0 msgr2=0x7f96c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96c00682f0 0x7f96c006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f96dc158df0 tx=0x7f96d8007ec0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 msgr2=0x7f96dc1e45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 0x7f96dc1e45e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009fd0 tx=0x7f96cc004750 comp rx=0 tx=0).stop 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 shutdown_connections 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f96dc159830 0x7f96dc0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96c00682f0 0x7f96c006a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96dc06c5f0 0x7f96dc1e45e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 --2- 172.21.15.83:0/698720882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96dc0b18a0 0x7f96dc1546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 >> 172.21.15.83:0/698720882 conn(0x7f96dc06e9a0 msgr2=0x7f96dc0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 shutdown_connections 2023-11-28T11:08:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.290+0000 7f96e3aa3700 1 -- 172.21.15.83:0/698720882 wait complete. 2023-11-28T11:08:58.303 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T11:08:58.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.422+0000 7fb8201cc700 1 Processor -- start 2023-11-28T11:08:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.422+0000 7fb8201cc700 1 -- start start 2023-11-28T11:08:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb8201cc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb81815a510 0x7fb8180cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb8201cc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8180cfdf0 0x7fb8180b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb8201cc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81815f1d0 con 0x7fb81815a8e0 2023-11-28T11:08:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb8201cc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8180cd780 con 0x7fb8180cfdf0 2023-11-28T11:08:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb8201cc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81815eef0 con 0x7fb81815a510 2023-11-28T11:08:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8180cfdf0 0x7fb8180b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81e769700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb81815a8e0 0x7fb8180b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47010/0 (socket says 172.21.15.83:47010) 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8180cfdf0 0x7fb8180b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49146/0 (socket says 172.21.15.83:49146) 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81e769700 1 -- 172.21.15.83:0/1712275390 learned_addr learned my addr 172.21.15.83:0/1712275390 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81df68700 1 --2- 172.21.15.83:0/1712275390 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb81815a510 0x7fb8180cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 -- 172.21.15.83:0/1712275390 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb81815a510 msgr2=0x7fb8180cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 --2- 172.21.15.83:0/1712275390 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb81815a510 0x7fb8180cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 -- 172.21.15.83:0/1712275390 >> v1:172.21.15.83:6789/0 conn(0x7fb81815a8e0 legacy=0x7fb8180b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 -- 172.21.15.83:0/1712275390 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8180cedd0 con 0x7fb8180cfdf0 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81d767700 1 --2- 172.21.15.83:0/1712275390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8180cfdf0 0x7fb8180b2960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb808009840 tx=0x7fb808009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7392f6e69cd510ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81cf66700 1 -- 172.21.15.83:0/1712275390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808011830 con 0x7fb8180cfdf0 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81cf66700 1 -- 172.21.15.83:0/1712275390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808011990 con 0x7fb8180cfdf0 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.423+0000 7fb81cf66700 1 -- 172.21.15.83:0/1712275390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808011d20 con 0x7fb8180cfdf0 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 -- 172.21.15.83:0/1712275390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8180cfdf0 msgr2=0x7fb8180b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 --2- 172.21.15.83:0/1712275390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8180cfdf0 0x7fb8180b2960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb808009840 tx=0x7fb808009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 -- 172.21.15.83:0/1712275390 shutdown_connections 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 --2- 172.21.15.83:0/1712275390 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb81815a510 0x7fb8180cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 --2- 172.21.15.83:0/1712275390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8180cfdf0 0x7fb8180b2960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 -- 172.21.15.83:0/1712275390 >> 172.21.15.83:0/1712275390 conn(0x7fb8180cd960 msgr2=0x7fb8180cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 -- 172.21.15.83:0/1712275390 shutdown_connections 2023-11-28T11:08:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.424+0000 7fb8201cc700 1 -- 172.21.15.83:0/1712275390 wait complete. 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.426+0000 7fb8201cc700 1 Processor -- start 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.426+0000 7fb8201cc700 1 -- start start 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.426+0000 7fb8201cc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8180cfdf0 0x7fb81814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.426+0000 7fb8201cc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb81815a510 0x7fb8181517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.426+0000 7fb8201cc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 0x7fb81806f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb8201cc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181efea0 con 0x7fb81815a8e0 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb8201cc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180c9940 con 0x7fb81815a510 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb8201cc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181522a0 con 0x7fb8180cfdf0 2023-11-28T11:08:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 0x7fb81806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 0x7fb81806f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49156/0 (socket says 172.21.15.83:49156) 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 -- 172.21.15.83:0/984275338 learned_addr learned my addr 172.21.15.83:0/984275338 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81d767700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb81815a510 0x7fb8181517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81df68700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8180cfdf0 0x7fb81814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 -- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8180cfdf0 msgr2=0x7fb81814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8180cfdf0 0x7fb81814dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 -- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb81815a510 msgr2=0x7fb8181517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb81815a510 0x7fb8181517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 -- 172.21.15.83:0/984275338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080094a0 con 0x7fb81815a8e0 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81df68700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8180cfdf0 0x7fb81814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb81e769700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 0x7fb81806f160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ca20 tx=0x7fb81000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000a980 con 0x7fb81815a8e0 2023-11-28T11:08:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000aae0 con 0x7fb81815a8e0 2023-11-28T11:08:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8180cf390 con 0x7fb81815a8e0 2023-11-28T11:08:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000ae50 con 0x7fb81815a8e0 2023-11-28T11:08:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.427+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8180cf0b0 con 0x7fb81815a8e0 2023-11-28T11:08:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.428+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8180b4440 con 0x7fb81815a8e0 2023-11-28T11:08:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.429+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb810019030 con 0x7fb81815a8e0 2023-11-28T11:08:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.429+0000 7fb80e7fc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb7fc0682f0 0x7fb7fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.429+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb810085850 con 0x7fb81815a8e0 2023-11-28T11:08:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.429+0000 7fb81df68700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb7fc0682f0 0x7fb7fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.429+0000 7fb81df68700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb7fc0682f0 0x7fb7fc06a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb81400aa50 tx=0x7fb814008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.433+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb81004b5e0 con 0x7fb81815a8e0 2023-11-28T11:08:58.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.617+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb818074320 con 0x7fb81815a8e0 2023-11-28T11:08:58.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.618+0000 7fb80e7fc700 1 -- 172.21.15.83:0/984275338 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fb810051e70 con 0x7fb81815a8e0 2023-11-28T11:08:58.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:08:58.620 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb7fc0682f0 msgr2=0x7fb7fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb7fc0682f0 0x7fb7fc06a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb81400aa50 tx=0x7fb814008040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 msgr2=0x7fb81806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 0x7fb81806f160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ca20 tx=0x7fb81000cd30 comp rx=0 tx=0).stop 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 shutdown_connections 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb7fc0682f0 0x7fb7fc06a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81815a8e0 0x7fb81806f160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8180cfdf0 0x7fb81814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.620+0000 7fb8201cc700 1 --2- 172.21.15.83:0/984275338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb81815a510 0x7fb8181517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.621+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 >> 172.21.15.83:0/984275338 conn(0x7fb8180cd960 msgr2=0x7fb8180b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.621+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 shutdown_connections 2023-11-28T11:08:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.621+0000 7fb8201cc700 1 -- 172.21.15.83:0/984275338 wait complete. 2023-11-28T11:08:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:08:58.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:58.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.757+0000 7fd531b1a700 1 Processor -- start 2023-11-28T11:08:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.757+0000 7fd531b1a700 1 -- start start 2023-11-28T11:08:58.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd531b1a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd531b1a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c06d4a0 con 0x7fd52c15c890 2023-11-28T11:08:58.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd531b1a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c0b5ae0 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd531b1a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0b5050 con 0x7fd52c06e370 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd52c15c890 0x7fd52c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47020/0 (socket says 172.21.15.83:47020) 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52b7fe700 1 -- 172.21.15.83:0/2755040068 learned_addr learned my addr 172.21.15.83:0/2755040068 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52affd700 1 --2- 172.21.15.83:0/2755040068 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207486842 0 0) 0x7fd52c06d4a0 con 0x7fd52c15c890 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5140035b0 con 0x7fd52c15c890 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192738587 0 0) 0x7fd52c0b5ae0 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c06d4a0 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52affd700 1 -- 172.21.15.83:0/2755040068 >> v1:172.21.15.90:6789/0 conn(0x7fd52c0b4bb0 legacy=0x7fd52c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 584718932 0 0) 0x7fd5140035b0 con 0x7fd52c15c890 2023-11-28T11:08:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52affd700 1 -- 172.21.15.83:0/2755040068 >> v1:172.21.15.83:6789/0 conn(0x7fd52c15c890 legacy=0x7fd52c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.758+0000 7fd52affd700 1 -- 172.21.15.83:0/2755040068 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c06c400 con 0x7fd52c06e370 2023-11-28T11:08:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd52affd700 1 --2- 172.21.15.83:0/2755040068 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c0aee60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009870 tx=0x7fd51c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ac5413fed93edc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5200017f0 con 0x7fd52c06e370 2023-11-28T11:08:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c004030 con 0x7fd52c06e370 2023-11-28T11:08:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd52a7fc700 1 -- 172.21.15.83:0/2755040068 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0043c0 con 0x7fd52c06e370 2023-11-28T11:08:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 -- 172.21.15.83:0/2755040068 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 msgr2=0x7fd52c0aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2755040068 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c0aee60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009870 tx=0x7fd51c009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 -- 172.21.15.83:0/2755040068 shutdown_connections 2023-11-28T11:08:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2755040068 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c0aee60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 -- 172.21.15.83:0/2755040068 >> 172.21.15.83:0/2755040068 conn(0x7fd52c0b2b60 msgr2=0x7fd52c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 -- 172.21.15.83:0/2755040068 shutdown_connections 2023-11-28T11:08:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.759+0000 7fd531b1a700 1 -- 172.21.15.83:0/2755040068 wait complete. 2023-11-28T11:08:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.760+0000 7fd531b1a700 1 Processor -- start 2023-11-28T11:08:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 -- start start 2023-11-28T11:08:58.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 0x7fd52c0c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd52c15c890 0x7fd52c0c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0c7ea0 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c15b480 con 0x7fd52c15c890 2023-11-28T11:08:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd531b1a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c0c4600 con 0x7fd52c06e370 2023-11-28T11:08:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.761+0000 7fd52affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 0x7fd52c0c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 0x7fd52c0c3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49166/0 (socket says 172.21.15.83:49166) 2023-11-28T11:08:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 -- 172.21.15.83:0/2044681460 learned_addr learned my addr 172.21.15.83:0/2044681460 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52b7fe700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52bfff700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd52c15c890 0x7fd52c0c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 -- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 msgr2=0x7fd52c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 -- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd52c15c890 msgr2=0x7fd52c0c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd52c15c890 0x7fd52c0c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 -- 172.21.15.83:0/2044681460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0094a0 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52bfff700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd52c15c890 0x7fd52c0c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd52affd700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 0x7fd52c0c3740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0090b0 tx=0x7fd51c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c003d80 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c0b5520 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c01f070 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c011870 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.762+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c0b5240 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.763+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c0b5800 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.763+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd51c02a020 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.764+0000 7fd513fff700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.764+0000 7fd52b7fe700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.764+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd51c087480 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.764+0000 7fd52b7fe700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd520001360 tx=0x7fd520000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.768+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd51c053970 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.951+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd52c0b3820 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.952+0000 7fd513fff700 1 -- 172.21.15.83:0/2044681460 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd51c056f90 con 0x7fd52c0b4bb0 2023-11-28T11:08:58.954 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:08:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 msgr2=0x7fd51806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd520001360 tx=0x7fd520000bc0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 msgr2=0x7fd52c0c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 0x7fd52c0c3740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0090b0 tx=0x7fd51c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 shutdown_connections 2023-11-28T11:08:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd52c15c890 0x7fd52c0c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd52c06e370 0x7fd52c15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 --2- 172.21.15.83:0/2044681460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd52c0b4bb0 0x7fd52c0c3740 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 >> 172.21.15.83:0/2044681460 conn(0x7fd52c0b2b60 msgr2=0x7fd52c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 shutdown_connections 2023-11-28T11:08:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:58.954+0000 7fd531b1a700 1 -- 172.21.15.83:0/2044681460 wait complete. 2023-11-28T11:08:58.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.090+0000 7f63d6e1a700 1 Processor -- start 2023-11-28T11:08:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.090+0000 7f63d6e1a700 1 -- start start 2023-11-28T11:08:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d6e1a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 0x7f63d0159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d6e1a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f63d0158cd0 0x7f63d00cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d6e1a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d00c9e70 con 0x7f63d00cb650 2023-11-28T11:08:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d6e1a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d015a000 con 0x7f63d0158cd0 2023-11-28T11:08:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d6e1a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d015c790 con 0x7f63d0158900 2023-11-28T11:08:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d4bb6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 0x7f63d0159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63cffff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f63d0158cd0 0x7f63d00cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d4bb6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 0x7f63d0159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:33374/0 (socket says 172.21.15.83:33374) 2023-11-28T11:08:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d4bb6700 1 -- 172.21.15.83:0/2126967124 learned_addr learned my addr 172.21.15.83:0/2126967124 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.091+0000 7f63d53b7700 1 --1- 172.21.15.83:0/2126967124 >> v1:172.21.15.90:6789/0 conn(0x7f63d00cb650 0x7f63d006d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45382/0 (socket says 172.21.15.83:45382) 2023-11-28T11:08:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63cf7fe700 1 -- 172.21.15.83:0/2126967124 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000928200 0 0) 0x7f63d00c9e70 con 0x7f63d00cb650 2023-11-28T11:08:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63cf7fe700 1 -- 172.21.15.83:0/2126967124 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c00035b0 con 0x7f63d00cb650 2023-11-28T11:08:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63d4bb6700 1 -- 172.21.15.83:0/2126967124 >> v2:172.21.15.90:3300/0 conn(0x7f63d0158cd0 msgr2=0x7f63d00cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63d4bb6700 1 --2- 172.21.15.83:0/2126967124 >> v2:172.21.15.90:3300/0 conn(0x7f63d0158cd0 0x7f63d00cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63d4bb6700 1 -- 172.21.15.83:0/2126967124 >> v1:172.21.15.90:6789/0 conn(0x7f63d00cb650 legacy=0x7f63d006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63d4bb6700 1 -- 172.21.15.83:0/2126967124 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d015ca70 con 0x7f63d0158900 2023-11-28T11:08:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63cffff700 1 --2- 172.21.15.83:0/2126967124 >> v2:172.21.15.90:3300/0 conn(0x7f63d0158cd0 0x7f63d00cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63d4bb6700 1 --2- 172.21.15.83:0/2126967124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 0x7f63d0159ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63c400bb90 tx=0x7f63c400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eabe01ed1600dab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63cf7fe700 1 -- 172.21.15.83:0/2126967124 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c400dd30 con 0x7f63d0158900 2023-11-28T11:08:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63cf7fe700 1 -- 172.21.15.83:0/2126967124 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c400de90 con 0x7f63d0158900 2023-11-28T11:08:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.092+0000 7f63cf7fe700 1 -- 172.21.15.83:0/2126967124 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4010690 con 0x7f63d0158900 2023-11-28T11:08:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 -- 172.21.15.83:0/2126967124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 msgr2=0x7f63d0159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/2126967124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 0x7f63d0159ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63c400bb90 tx=0x7f63c400bea0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 -- 172.21.15.83:0/2126967124 shutdown_connections 2023-11-28T11:08:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/2126967124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158900 0x7f63d0159ac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/2126967124 >> v2:172.21.15.90:3300/0 conn(0x7f63d0158cd0 0x7f63d00cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 -- 172.21.15.83:0/2126967124 >> 172.21.15.83:0/2126967124 conn(0x7f63d00b1090 msgr2=0x7f63d00b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 -- 172.21.15.83:0/2126967124 shutdown_connections 2023-11-28T11:08:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.093+0000 7f63d6e1a700 1 -- 172.21.15.83:0/2126967124 wait complete. 2023-11-28T11:08:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.094+0000 7f63d6e1a700 1 Processor -- start 2023-11-28T11:08:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.094+0000 7f63d6e1a700 1 -- start start 2023-11-28T11:08:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d6e1a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d00cb650 0x7f63d0154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d6e1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 0x7f63d014f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d6e1a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158cd0 0x7f63d015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d6e1a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d014f630 con 0x7f63d0158900 2023-11-28T11:08:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d6e1a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0154d90 con 0x7f63d00cb650 2023-11-28T11:08:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d6e1a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0155070 con 0x7f63d0158cd0 2023-11-28T11:08:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 0x7f63d014f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d4bb6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d00cb650 0x7f63d0154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63d53b7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158cd0 0x7f63d015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 0x7f63d014f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49182/0 (socket says 172.21.15.83:49182) 2023-11-28T11:08:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 -- 172.21.15.83:0/3782679120 learned_addr learned my addr 172.21.15.83:0/3782679120 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 -- 172.21.15.83:0/3782679120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158cd0 msgr2=0x7f63d015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158cd0 0x7f63d015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 -- 172.21.15.83:0/3782679120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d00cb650 msgr2=0x7f63d0154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d00cb650 0x7f63d0154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.095+0000 7f63cffff700 1 -- 172.21.15.83:0/3782679120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0094a0 con 0x7f63d0158900 2023-11-28T11:08:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.096+0000 7f63cffff700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 0x7f63d014f0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00ebe0 tx=0x7f63bc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.096+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00c850 con 0x7f63d0158900 2023-11-28T11:08:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.096+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c400b7f0 con 0x7f63d0158900 2023-11-28T11:08:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.096+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00c9b0 con 0x7f63d0158900 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.096+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00cd40 con 0x7f63d0158900 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.096+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d015d030 con 0x7f63d0158900 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.097+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d015cd50 con 0x7f63d0158900 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.097+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f63bc023070 con 0x7f63d0158900 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.097+0000 7f63cd7fa700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f63b40682f0 0x7f63b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.098+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63bc0863d0 con 0x7f63d0158900 2023-11-28T11:08:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.098+0000 7f63d4bb6700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f63b40682f0 0x7f63b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.098+0000 7f63d4bb6700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f63b40682f0 0x7f63b406a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63c400bee0 tx=0x7f63c4013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.102+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f63bc0527d0 con 0x7f63d0158900 2023-11-28T11:08:59.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.286+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63d00c36e0 con 0x7f63d0158900 2023-11-28T11:08:59.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.287+0000 7f63cd7fa700 1 -- 172.21.15.83:0/3782679120 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63bc00cea0 con 0x7f63d0158900 2023-11-28T11:08:59.288 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:08:59.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f63b40682f0 msgr2=0x7f63b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f63b40682f0 0x7f63b406a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63c400bee0 tx=0x7f63c4013000 comp rx=0 tx=0).stop 2023-11-28T11:08:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 msgr2=0x7f63d014f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 0x7f63d014f0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00ebe0 tx=0x7f63bc00eef0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 shutdown_connections 2023-11-28T11:08:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f63d0158cd0 0x7f63d015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d00cb650 0x7f63d0154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f63b40682f0 0x7f63b406a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 --2- 172.21.15.83:0/3782679120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63d0158900 0x7f63d014f0f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 >> 172.21.15.83:0/3782679120 conn(0x7f63d00b1090 msgr2=0x7f63d00cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 shutdown_connections 2023-11-28T11:08:59.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.289+0000 7f63d6e1a700 1 -- 172.21.15.83:0/3782679120 wait complete. 2023-11-28T11:08:59.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:59.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.424+0000 7fc4aab57700 1 Processor -- start 2023-11-28T11:08:59.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.424+0000 7fc4aab57700 1 -- start start 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4aab57700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415e5c0 0x7fc4a415e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4aab57700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a4151a30 con 0x7fc4a40d11f0 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4aab57700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a40cb460 con 0x7fc4a415b3b0 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4aab57700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40cb740 con 0x7fc4a415e5c0 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a90f4700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc4a40d11f0 0x7fc4a40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47050/0 (socket says 172.21.15.83:47050) 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a90f4700 1 -- 172.21.15.83:0/665803665 learned_addr learned my addr 172.21.15.83:0/665803665 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a88f3700 1 --2- 172.21.15.83:0/665803665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415e5c0 0x7fc4a415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474299453 0 0) 0x7fc4a4151a30 con 0x7fc4a40d11f0 2023-11-28T11:08:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4940035b0 con 0x7fc4a40d11f0 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775340003 0 0) 0x7fc4a40cb460 con 0x7fc4a415b3b0 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a4151a30 con 0x7fc4a415b3b0 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a88f3700 1 -- 172.21.15.83:0/665803665 >> v1:172.21.15.90:6789/0 conn(0x7fc4a415b3b0 legacy=0x7fc4a40d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a88f3700 1 -- 172.21.15.83:0/665803665 >> v1:172.21.15.83:6789/0 conn(0x7fc4a40d11f0 legacy=0x7fc4a40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.425+0000 7fc4a88f3700 1 -- 172.21.15.83:0/665803665 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a415abb0 con 0x7fc4a415e5c0 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4a88f3700 1 --2- 172.21.15.83:0/665803665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415e5c0 0x7fc4a415e990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc498009870 tx=0x7fc498009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6eb74053c957ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011830 con 0x7fc4a415e5c0 2023-11-28T11:08:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498011990 con 0x7fc4a415e5c0 2023-11-28T11:08:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4a37fe700 1 -- 172.21.15.83:0/665803665 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011d20 con 0x7fc4a415e5c0 2023-11-28T11:08:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 -- 172.21.15.83:0/665803665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415e5c0 msgr2=0x7fc4a415e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 --2- 172.21.15.83:0/665803665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415e5c0 0x7fc4a415e990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc498009870 tx=0x7fc498009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 -- 172.21.15.83:0/665803665 shutdown_connections 2023-11-28T11:08:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 --2- 172.21.15.83:0/665803665 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415e5c0 0x7fc4a415e990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 -- 172.21.15.83:0/665803665 >> 172.21.15.83:0/665803665 conn(0x7fc4a406ea50 msgr2=0x7fc4a4070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 -- 172.21.15.83:0/665803665 shutdown_connections 2023-11-28T11:08:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.426+0000 7fc4aab57700 1 -- 172.21.15.83:0/665803665 wait complete. 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.428+0000 7fc4aab57700 1 Processor -- start 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.428+0000 7fc4aab57700 1 -- start start 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4aab57700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415b3b0 0x7fc4a414b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4aab57700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4a415e5c0 0x7fc4a4151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4aab57700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 0x7fc4a41525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4aab57700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4152f40 con 0x7fc4a4152100 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4aab57700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40b5e10 con 0x7fc4a415e5c0 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4aab57700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4152c60 con 0x7fc4a415b3b0 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 0x7fc4a41525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 0x7fc4a41525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49196/0 (socket says 172.21.15.83:49196) 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 -- 172.21.15.83:0/4269390305 learned_addr learned my addr 172.21.15.83:0/4269390305 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a3fff700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4a415e5c0 0x7fc4a4151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a88f3700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415b3b0 0x7fc4a414b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 -- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415b3b0 msgr2=0x7fc4a414b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415b3b0 0x7fc4a414b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 -- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4a415e5c0 msgr2=0x7fc4a4151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4a415e5c0 0x7fc4a4151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a90f4700 1 -- 172.21.15.83:0/4269390305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980094a0 con 0x7fc4a4152100 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a3fff700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4a415e5c0 0x7fc4a4151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.429+0000 7fc4a88f3700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415b3b0 0x7fc4a414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.430+0000 7fc4a90f4700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 0x7fc4a41525a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00ca80 tx=0x7fc49c00ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.430+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00aa60 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.430+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00abc0 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.430+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a415eed0 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.430+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c012400 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.430+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a415ae90 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.431+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a40cba20 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.431+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc49c019030 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.431+0000 7fc4a17fa700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4880682f0 0x7fc48806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.432+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc49c085b40 con 0x7fc4a4152100 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.432+0000 7fc4a88f3700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4880682f0 0x7fc48806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.432+0000 7fc4a88f3700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4880682f0 0x7fc48806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4980090b0 tx=0x7fc49801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.436+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc49c0520b0 con 0x7fc4a4152100 2023-11-28T11:08:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.620+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4a40b5620 con 0x7fc4a4152100 2023-11-28T11:08:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.620+0000 7fc4a17fa700 1 -- 172.21.15.83:0/4269390305 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc49c017020 con 0x7fc4a4152100 2023-11-28T11:08:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.622+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4880682f0 msgr2=0x7fc48806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.622+0000 7fc4aab57700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4880682f0 0x7fc48806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4980090b0 tx=0x7fc49801c040 comp rx=0 tx=0).stop 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.622+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 msgr2=0x7fc4a41525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.622+0000 7fc4aab57700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 0x7fc4a41525a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00ca80 tx=0x7fc49c00ce40 comp rx=0 tx=0).stop 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.622+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 shutdown_connections 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4880682f0 0x7fc48806a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a4152100 0x7fc4a41525a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc4a415b3b0 0x7fc4a414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 --2- 172.21.15.83:0/4269390305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4a415e5c0 0x7fc4a4151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 >> 172.21.15.83:0/4269390305 conn(0x7fc4a40d11f0 msgr2=0x7fc4a4070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 shutdown_connections 2023-11-28T11:08:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.623+0000 7fc4aab57700 1 -- 172.21.15.83:0/4269390305 wait complete. 2023-11-28T11:08:59.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.760+0000 7f7491017700 1 Processor -- start 2023-11-28T11:08:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.760+0000 7f7491017700 1 -- start start 2023-11-28T11:08:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7491017700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f748c0aee60 0x7f748c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7491017700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c06d4a0 con 0x7f748c15c890 2023-11-28T11:08:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7491017700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c0b5ae0 con 0x7f748c06e370 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7491017700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c06c400 con 0x7f748c0aee60 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748b59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f748c0aee60 0x7f748c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748ad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f748c15c890 0x7f748c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47060/0 (socket says 172.21.15.83:47060) 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748b59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f748c0aee60 0x7f748c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49204/0 (socket says 172.21.15.83:49204) 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748ad9d700 1 -- 172.21.15.83:0/1720313942 learned_addr learned my addr 172.21.15.83:0/1720313942 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748b59e700 1 -- 172.21.15.83:0/1720313942 >> v1:172.21.15.90:6789/0 conn(0x7f748c06e370 legacy=0x7f748c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748b59e700 1 -- 172.21.15.83:0/1720313942 >> v1:172.21.15.83:6789/0 conn(0x7f748c15c890 legacy=0x7f748c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7489d9b700 1 -- 172.21.15.83:0/1720313942 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004125058 0 0) 0x7f748c06d4a0 con 0x7f748c15c890 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748b59e700 1 -- 172.21.15.83:0/1720313942 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0b5240 con 0x7f748c0aee60 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f748b59e700 1 --2- 172.21.15.83:0/1720313942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c0aee60 0x7f748c0b4bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7484009d90 tx=0x7f748400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5215be59dab26ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7489d9b700 1 -- 172.21.15.83:0/1720313942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400e070 con 0x7f748c0aee60 2023-11-28T11:08:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.761+0000 7f7489d9b700 1 -- 172.21.15.83:0/1720313942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400bb90 con 0x7f748c0aee60 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7489d9b700 1 -- 172.21.15.83:0/1720313942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484003a20 con 0x7f748c0aee60 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 -- 172.21.15.83:0/1720313942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c0aee60 msgr2=0x7f748c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 --2- 172.21.15.83:0/1720313942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c0aee60 0x7f748c0b4bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7484009d90 tx=0x7f748400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 -- 172.21.15.83:0/1720313942 shutdown_connections 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 --2- 172.21.15.83:0/1720313942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c0aee60 0x7f748c0b4bb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 -- 172.21.15.83:0/1720313942 >> 172.21.15.83:0/1720313942 conn(0x7f748c0b2b60 msgr2=0x7f748c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 -- 172.21.15.83:0/1720313942 shutdown_connections 2023-11-28T11:08:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.762+0000 7f7491017700 1 -- 172.21.15.83:0/1720313942 wait complete. 2023-11-28T11:08:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 Processor -- start 2023-11-28T11:08:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 -- start start 2023-11-28T11:08:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f748c06e370 0x7f748c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f748c0aee60 0x7f748c0c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 0x7f748c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c155fd0 con 0x7f748c15c890 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c156410 con 0x7f748c06e370 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.764+0000 7f7491017700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0c3890 con 0x7f748c0aee60 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 0x7f748c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 0x7f748c15cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49210/0 (socket says 172.21.15.83:49210) 2023-11-28T11:08:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 -- 172.21.15.83:0/2335255051 learned_addr learned my addr 172.21.15.83:0/2335255051 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748ad9d700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f748c06e370 0x7f748c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748a59c700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f748c0aee60 0x7f748c0c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 -- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f748c0aee60 msgr2=0x7f748c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f748c0aee60 0x7f748c0c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 -- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f748c06e370 msgr2=0x7f748c155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f748c06e370 0x7f748c155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 -- 172.21.15.83:0/2335255051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009a70 con 0x7f748c15c890 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f748b59e700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 0x7f748c15cda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f748c08a420 tx=0x7f7484004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484012630 con 0x7f748c15c890 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74840127b0 con 0x7f748c15c890 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0b5800 con 0x7f748c15c890 2023-11-28T11:08:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484012b60 con 0x7f748c15c890 2023-11-28T11:08:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.765+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0b5520 con 0x7f748c15c890 2023-11-28T11:08:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.766+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c06ac30 con 0x7f748c15c890 2023-11-28T11:08:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.767+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7484012cc0 con 0x7f748c15c890 2023-11-28T11:08:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.767+0000 7f74737fe700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74780682f0 0x7f747806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.767+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7484088440 con 0x7f748c15c890 2023-11-28T11:08:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.767+0000 7f748ad9d700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74780682f0 0x7f747806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.767+0000 7f748ad9d700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74780682f0 0x7f747806a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7480007fd0 tx=0x7f7480009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.771+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74840549b0 con 0x7f748c15c890 2023-11-28T11:08:59.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.957+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f748c0d0230 con 0x7f748c15c890 2023-11-28T11:08:59.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.957+0000 7f74737fe700 1 -- 172.21.15.83:0/2335255051 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7484057fd0 con 0x7f748c15c890 2023-11-28T11:08:59.959 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:08:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74780682f0 msgr2=0x7f747806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74780682f0 0x7f747806a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7480007fd0 tx=0x7f7480009040 comp rx=0 tx=0).stop 2023-11-28T11:08:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 msgr2=0x7f748c15cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 0x7f748c15cda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f748c08a420 tx=0x7f7484004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 shutdown_connections 2023-11-28T11:08:59.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74780682f0 0x7f747806a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f748c15c890 0x7f748c15cda0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f748c06e370 0x7f748c155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 --2- 172.21.15.83:0/2335255051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f748c0aee60 0x7f748c0c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 >> 172.21.15.83:0/2335255051 conn(0x7f748c0b2b60 msgr2=0x7f748c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 shutdown_connections 2023-11-28T11:08:59.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:08:59.959+0000 7f7491017700 1 -- 172.21.15.83:0/2335255051 wait complete. 2023-11-28T11:08:59.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:00.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.096+0000 7f0929809700 1 Processor -- start 2023-11-28T11:09:00.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.096+0000 7f0929809700 1 -- start start 2023-11-28T11:09:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0929809700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09240aee60 0x7f09240b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0929809700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092406d4a0 con 0x7f092406e370 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0929809700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09240b5ae0 con 0x7f092415c890 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0929809700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092406c400 con 0x7f09240aee60 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09237fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09240aee60 0x7f09240b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09227fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f092406e370 0x7f092406b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47076/0 (socket says 172.21.15.83:47076) 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09227fc700 1 -- 172.21.15.83:0/269612137 learned_addr learned my addr 172.21.15.83:0/269612137 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09237fe700 1 -- 172.21.15.83:0/269612137 >> v1:172.21.15.90:6789/0 conn(0x7f092415c890 legacy=0x7f092406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09237fe700 1 -- 172.21.15.83:0/269612137 >> v1:172.21.15.83:6789/0 conn(0x7f092406e370 legacy=0x7f092406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0921ffb700 1 -- 172.21.15.83:0/269612137 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882868979 0 0) 0x7f092406d4a0 con 0x7f092406e370 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09237fe700 1 -- 172.21.15.83:0/269612137 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240b5240 con 0x7f09240aee60 2023-11-28T11:09:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f09237fe700 1 --2- 172.21.15.83:0/269612137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09240aee60 0x7f09240b4bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f091c009d90 tx=0x7f091c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d21f720ad496d34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0921ffb700 1 -- 172.21.15.83:0/269612137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00e070 con 0x7f09240aee60 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.097+0000 7f0921ffb700 1 -- 172.21.15.83:0/269612137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c00bbd0 con 0x7f09240aee60 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0921ffb700 1 -- 172.21.15.83:0/269612137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c012430 con 0x7f09240aee60 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 -- 172.21.15.83:0/269612137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09240aee60 msgr2=0x7f09240b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 --2- 172.21.15.83:0/269612137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09240aee60 0x7f09240b4bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f091c009d90 tx=0x7f091c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 -- 172.21.15.83:0/269612137 shutdown_connections 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 --2- 172.21.15.83:0/269612137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09240aee60 0x7f09240b4bb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 -- 172.21.15.83:0/269612137 >> 172.21.15.83:0/269612137 conn(0x7f09240b2b60 msgr2=0x7f09240b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 -- 172.21.15.83:0/269612137 shutdown_connections 2023-11-28T11:09:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.098+0000 7f0929809700 1 -- 172.21.15.83:0/269612137 wait complete. 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.099+0000 7f0929809700 1 Processor -- start 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.099+0000 7f0929809700 1 -- start start 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f0929809700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f092406e370 0x7f09240c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f0929809700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09240aee60 0x7f09240cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f0929809700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 0x7f09240c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f0929809700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240c48e0 con 0x7f092415c890 2023-11-28T11:09:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f0929809700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240c4bc0 con 0x7f09240aee60 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f0929809700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240c4ea0 con 0x7f092406e370 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f09237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 0x7f09240c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f09237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 0x7f09240c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49224/0 (socket says 172.21.15.83:49224) 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f09237fe700 1 -- 172.21.15.83:0/2307705087 learned_addr learned my addr 172.21.15.83:0/2307705087 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.100+0000 7f09227fc700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09240aee60 0x7f09240cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f0922ffd700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f092406e370 0x7f09240c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f09237fe700 1 -- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f092406e370 msgr2=0x7f09240c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f09237fe700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f092406e370 0x7f09240c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f09237fe700 1 -- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09240aee60 msgr2=0x7f09240cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f09237fe700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09240aee60 0x7f09240cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f09237fe700 1 -- 172.21.15.83:0/2307705087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c009a70 con 0x7f092415c890 2023-11-28T11:09:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f0922ffd700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f092406e370 0x7f09240c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f09237fe700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 0x7f09240c5680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f091c014040 tx=0x7f091c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c01b810 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c020070 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09240b5800 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c01bbb0 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.101+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09240b5520 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.102+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092406ac30 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.102+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f091c01bd10 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.103+0000 7f090bfff700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f09100682f0 0x7f091006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.103+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f091c022070 con 0x7f092415c890 2023-11-28T11:09:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.103+0000 7f0922ffd700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f09100682f0 0x7f091006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.103+0000 7f0922ffd700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f09100682f0 0x7f091006a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0918000fb0 tx=0x7f0918009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.107+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f091c054b50 con 0x7f092415c890 2023-11-28T11:09:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.289+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f09240c54f0 con 0x7f092415c890 2023-11-28T11:09:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.289+0000 7f090bfff700 1 -- 172.21.15.83:0/2307705087 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f091c058170 con 0x7f092415c890 2023-11-28T11:09:00.291 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f09100682f0 msgr2=0x7f091006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f09100682f0 0x7f091006a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0918000fb0 tx=0x7f0918009040 comp rx=0 tx=0).stop 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 msgr2=0x7f09240c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 0x7f09240c5680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f091c014040 tx=0x7f091c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 shutdown_connections 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f09100682f0 0x7f091006a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092415c890 0x7f09240c5680 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f092406e370 0x7f09240c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 --2- 172.21.15.83:0/2307705087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09240aee60 0x7f09240cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 >> 172.21.15.83:0/2307705087 conn(0x7f09240b2b60 msgr2=0x7f092406b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 shutdown_connections 2023-11-28T11:09:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.291+0000 7f0929809700 1 -- 172.21.15.83:0/2307705087 wait complete. 2023-11-28T11:09:00.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.422+0000 7f6ea43b8700 1 Processor -- start 2023-11-28T11:09:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.423+0000 7f6ea43b8700 1 -- start start 2023-11-28T11:09:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.423+0000 7f6ea43b8700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6e9c15c850 0x7f6e9c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea43b8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c06ff80 con 0x7f6e9c15cc20 2023-11-28T11:09:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea43b8700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c161450 con 0x7f6e9c06cf90 2023-11-28T11:09:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea43b8700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0ce630 con 0x7f6e9c15c850 2023-11-28T11:09:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1953700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6e9c15cc20 0x7f6e9c06f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47090/0 (socket says 172.21.15.83:47090) 2023-11-28T11:09:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1953700 1 -- 172.21.15.83:0/749904353 learned_addr learned my addr 172.21.15.83:0/749904353 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea2154700 1 --2- 172.21.15.83:0/749904353 >> v2:172.21.15.90:3300/0 conn(0x7f6e9c15c850 0x7f6e9c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993180032 0 0) 0x7f6e9c06ff80 con 0x7f6e9c15cc20 2023-11-28T11:09:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e840035b0 con 0x7f6e9c15cc20 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246778404 0 0) 0x7f6e840035b0 con 0x7f6e9c15cc20 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 >> v2:172.21.15.90:3300/0 conn(0x7f6e9c15c850 msgr2=0x7f6e9c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 --2- 172.21.15.83:0/749904353 >> v2:172.21.15.90:3300/0 conn(0x7f6e9c15c850 0x7f6e9c0ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 >> v1:172.21.15.90:6789/0 conn(0x7f6e9c06cf90 legacy=0x7f6e9c06f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c1608d0 con 0x7f6e9c15cc20 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6e90001670 con 0x7f6e9c15cc20 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 >> v1:172.21.15.83:6789/0 conn(0x7f6e9c15cc20 legacy=0x7f6e9c06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e84003a00 0x7f6e84003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e84004850 0x7f6e84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84005200 0x7f6e8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0ce630 con 0x7f6e84004850 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8400c110 con 0x7f6e84003a00 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8400c2d0 con 0x7f6e84005200 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea1953700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e84004850 0x7f6e84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea2154700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e84003a00 0x7f6e84003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.424+0000 7f6ea2955700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84005200 0x7f6e8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1953700 1 -- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84005200 msgr2=0x7f6e8400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1953700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84005200 0x7f6e8400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1953700 1 -- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e84003a00 msgr2=0x7f6e84003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1953700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e84003a00 0x7f6e84003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1953700 1 -- 172.21.15.83:0/749904353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840035b0 con 0x7f6e84004850 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea2955700 1 -- 172.21.15.83:0/749904353 reap_dead start 2023-11-28T11:09:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1953700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e84004850 0x7f6e84004cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c240 tx=0x7f6e84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684125d7f9342b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90007c10 con 0x7f6e84004850 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9000cde0 con 0x7f6e84004850 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea1152700 1 -- 172.21.15.83:0/749904353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90005ba0 con 0x7f6e84004850 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea43b8700 1 -- 172.21.15.83:0/749904353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e84004850 msgr2=0x7f6e84004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e84004850 0x7f6e84004cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c240 tx=0x7f6e84005880 comp rx=0 tx=0).stop 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.425+0000 7f6ea43b8700 1 -- 172.21.15.83:0/749904353 shutdown_connections 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.426+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/749904353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e84004850 0x7f6e84004cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.426+0000 7f6ea43b8700 1 -- 172.21.15.83:0/749904353 >> 172.21.15.83:0/749904353 conn(0x7f6e9c0cd960 msgr2=0x7f6e9c0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.426+0000 7f6ea43b8700 1 -- 172.21.15.83:0/749904353 shutdown_connections 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.426+0000 7f6ea43b8700 1 -- 172.21.15.83:0/749904353 wait complete. 2023-11-28T11:09:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 Processor -- start 2023-11-28T11:09:00.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 -- start start 2023-11-28T11:09:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84004850 0x7f6e9c1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e9c1573c0 0x7f6e9c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 0x7f6e9c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0c4840 con 0x7f6e9c157830 2023-11-28T11:09:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c1591a0 con 0x7f6e9c1573c0 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.427+0000 7f6ea43b8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15a810 con 0x7f6e84004850 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 0x7f6e9c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2154700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84004850 0x7f6e9c1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 0x7f6e9c15a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49244/0 (socket says 172.21.15.83:49244) 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 -- 172.21.15.83:0/3318283297 learned_addr learned my addr 172.21.15.83:0/3318283297 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea1953700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e9c1573c0 0x7f6e9c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 -- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84004850 msgr2=0x7f6e9c1f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84004850 0x7f6e9c1f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 -- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e9c1573c0 msgr2=0x7f6e9c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e9c1573c0 0x7f6e9c159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 -- 172.21.15.83:0/3318283297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c160e90 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea2955700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 0x7f6e9c15a1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c04e880 tx=0x7f6e9400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94003c10 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94003d90 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c160bb0 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400d950 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.428+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c161170 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.429+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c074320 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.429+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e9400dab0 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.430+0000 7f6e9affd700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e880682f0 0x7f6e8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.430+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6e940861f0 con 0x7f6e9c157830 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.430+0000 7f6ea2154700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e880682f0 0x7f6e8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.430+0000 7f6ea2154700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e880682f0 0x7f6e8806a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0688a0 tx=0x7f6e8c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.434+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e94052760 con 0x7f6e9c157830 2023-11-28T11:09:00.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.658+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e9c0b6e60 con 0x7f6e9c157830 2023-11-28T11:09:00.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.658+0000 7f6e9affd700 1 -- 172.21.15.83:0/3318283297 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f6e94055d80 con 0x7f6e9c157830 2023-11-28T11:09:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:00.660 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:51.826077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e880682f0 msgr2=0x7f6e8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e880682f0 0x7f6e8806a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0688a0 tx=0x7f6e8c009210 comp rx=0 tx=0).stop 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 msgr2=0x7f6e9c15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 0x7f6e9c15a1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c04e880 tx=0x7f6e9400d150 comp rx=0 tx=0).stop 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 shutdown_connections 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e880682f0 0x7f6e8806a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e9c157830 0x7f6e9c15a1e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e84004850 0x7f6e9c1f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 --2- 172.21.15.83:0/3318283297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e9c1573c0 0x7f6e9c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 >> 172.21.15.83:0/3318283297 conn(0x7f6e9c0cd960 msgr2=0x7f6e9c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 shutdown_connections 2023-11-28T11:09:00.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:00.660+0000 7f6ea43b8700 1 -- 172.21.15.83:0/3318283297 wait complete. 2023-11-28T11:09:00.663 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2023-11-28T11:09:01.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:08:35.518363+0000', 'modified': '2023-11-28T11:08:51.826077+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/3706545143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 3706545143}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 3706545143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:09:01.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:09:01.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T11:09:01.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.198+0000 7f3e41d6f700 1 Processor -- start 2023-11-28T11:09:01.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.198+0000 7f3e41d6f700 1 -- start start 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e41d6f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e3c0aee20 0x7f3e3c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e41d6f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c0b5b00 con 0x7f3e3c0af1f0 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e41d6f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c06c800 con 0x7f3e3c06c370 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e41d6f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c06cae0 con 0x7f3e3c0aee20 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e3b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e3c0aee20 0x7f3e3c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e3bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3e3c0af1f0 0x7f3e3c0cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47112/0 (socket says 172.21.15.83:47112) 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e3bfff700 1 -- 172.21.15.83:0/1872358472 learned_addr learned my addr 172.21.15.83:0/1872358472 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/1872358472 >> v1:172.21.15.90:6789/0 conn(0x7f3e3c06c370 legacy=0x7f3e3c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/1872358472 >> v1:172.21.15.83:6789/0 conn(0x7f3e3c0af1f0 legacy=0x7f3e3c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.199+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/1872358472 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c06cdc0 con 0x7f3e3c0aee20 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/1872358472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c0b55c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009820 tx=0x7f3e24009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb96472b1fdfa947 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e3a7fc700 1 -- 172.21.15.83:0/1872358472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24005e30 con 0x7f3e3c0aee20 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e3a7fc700 1 -- 172.21.15.83:0/1872358472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e24006030 con 0x7f3e3c0aee20 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e3a7fc700 1 -- 172.21.15.83:0/1872358472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240063e0 con 0x7f3e3c0aee20 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 -- 172.21.15.83:0/1872358472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c0aee20 msgr2=0x7f3e3c0b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/1872358472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c0b55c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009820 tx=0x7f3e24009b30 comp rx=0 tx=0).stop 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 -- 172.21.15.83:0/1872358472 shutdown_connections 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/1872358472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c0b55c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 -- 172.21.15.83:0/1872358472 >> 172.21.15.83:0/1872358472 conn(0x7f3e3c0b2b60 msgr2=0x7f3e3c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 -- 172.21.15.83:0/1872358472 shutdown_connections 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.200+0000 7f3e41d6f700 1 -- 172.21.15.83:0/1872358472 wait complete. 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.201+0000 7f3e41d6f700 1 Processor -- start 2023-11-28T11:09:01.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 -- start start 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 0x7f3e3c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e3c0af1f0 0x7f3e3c157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c153d10 con 0x7f3e3c06c370 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c1530d0 con 0x7f3e3c0aee20 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e41d6f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15cad0 con 0x7f3e3c0af1f0 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 0x7f3e3c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 0x7f3e3c14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49254/0 (socket says 172.21.15.83:49254) 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.202+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/4223087094 learned_addr learned my addr 172.21.15.83:0/4223087094 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e3c0af1f0 0x7f3e3c157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3affd700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e3c0af1f0 msgr2=0x7f3e3c157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e3c0af1f0 0x7f3e3c157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c0aee20 msgr2=0x7f3e3c15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c15c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/4223087094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24009430 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 0x7f3e3c14ec50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e24000c00 tx=0x7f3e24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e33fff700 1 -- 172.21.15.83:0/4223087094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24004250 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e33fff700 1 -- 172.21.15.83:0/4223087094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e240043d0 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c06fc70 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e33fff700 1 -- 172.21.15.83:0/4223087094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240036a0 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.203+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c06d0a0 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.204+0000 7f3e33fff700 1 -- 172.21.15.83:0/4223087094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e2400f050 con 0x7f3e3c06c370 2023-11-28T11:09:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.205+0000 7f3e33fff700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3e1c0682f0 0x7f3e1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.205+0000 7f3e33fff700 1 -- 172.21.15.83:0/4223087094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e24089f30 con 0x7f3e3c06c370 2023-11-28T11:09:01.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.205+0000 7f3e3affd700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3e1c0682f0 0x7f3e1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.205+0000 7f3e3affd700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3e1c0682f0 0x7f3e1c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b0960 tx=0x7f3e2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.207+0000 7f3e4def8b80 1 Processor -- start 2023-11-28T11:09:01.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.207+0000 7f3e4def8b80 1 -- start start 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.207+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 0x55dd7ad36a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.207+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad5e130 0x55dd7ad60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.207+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad7c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ac20bc0 con 0x55dd7ad60810 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ac8f350 con 0x55dd7ad5e130 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ac88880 con 0x55dd7ad61120 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 0x55dd7ad36a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 0x55dd7ad36a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49262/0 (socket says 172.21.15.83:49262) 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3133002334 learned_addr learned my addr 172.21.15.83:0/3133002334 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad7c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3affd700 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad5e130 0x55dd7ad60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 msgr2=0x55dd7ad7c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad7c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad5e130 msgr2=0x55dd7ad60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad5e130 0x55dd7ad60be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3133002334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd7ab0a5e0 con 0x55dd7ad60810 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3affd700 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad5e130 0x55dd7ad60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 0x55dd7ad36a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e24059a80 tx=0x7f3e24059ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b314e1757385fe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e317fa700 1 -- 172.21.15.83:0/3133002334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24071100 con 0x55dd7ad60810 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e317fa700 1 -- 172.21.15.83:0/3133002334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e24071260 con 0x55dd7ad60810 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.208+0000 7f3e317fa700 1 -- 172.21.15.83:0/3133002334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24061e90 con 0x55dd7ad60810 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3133002334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 msgr2=0x55dd7ad36a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 0x55dd7ad36a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e24059a80 tx=0x7f3e24059ab0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3133002334 shutdown_connections 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad7c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad60810 0x55dd7ad36a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3133002334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad5e130 0x55dd7ad60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3133002334 >> 172.21.15.83:0/3133002334 conn(0x55dd7ad33d60 msgr2=0x55dd7ad346f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3133002334 shutdown_connections 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.209+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3133002334 wait complete. 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 -- start start 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 Processor -- start 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad5e130 0x55dd7ad56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad5f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 0x55dd7ad5fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ab0c8d0 con 0x55dd7ad61120 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad7ff00 con 0x55dd7ad60810 2023-11-28T11:09:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad5d9a0 con 0x55dd7ad5e130 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.210+0000 7f3e4def8b80 10 client.0 _ll_register_callbacks cb 0x55dd7ab2e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 0x55dd7ad5fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad5e130 0x55dd7ad56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 0x55dd7ad5fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49272/0 (socket says 172.21.15.83:49272) 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad5f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 -- 172.21.15.83:0/2045686996 learned_addr learned my addr 172.21.15.83:0/2045686996 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 -- 172.21.15.83:0/2045686996 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad5e130 msgr2=0x55dd7ad56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad5e130 0x55dd7ad56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 -- 172.21.15.83:0/2045686996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 msgr2=0x55dd7ad5f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad5f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 -- 172.21.15.83:0/2045686996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c06d0a0 con 0x55dd7ad61120 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 0x55dd7ad5fb30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f820 tx=0x7f3e3400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ecce1fe1f2ee3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e1a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/2045686996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340072e0 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/2045686996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e34007440 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd7ac56140 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/2045686996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e34001b10 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e4def8b80 10 client.4788 fetch_fsmap learned FSMap version 9 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55dd7aab6d90 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/2045686996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dd7ac56140 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.211+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/2045686996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3e34012070 con 0x55dd7ad61120 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 10 client.4788 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 10 client.4788 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 20 client.4788 populate_metadata read hostname 'smithi083' 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad58b30 0x55dd7ad8ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 4 client.4788 mds_command: new command op to 4263 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 --> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd7ab0cf10 con 0x55dd7ad58b30 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad58b30 0x55dd7ad8ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e1a7fc700 10 client.4788 ms_handle_connect on v2:172.21.15.83:6835/3706545143 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad58b30 0x55dd7ad8ec20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29c8a3fdc01ccf7 server_cookie=b02dd699621356c7 in_seq=0 out_seq=0 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/2045686996 <== mds.0 v2:172.21.15.83:6835/3706545143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55dd7ab0cf10 con 0x55dd7ad58b30 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e1a7fc700 10 client.4788 handle_command_reply: tid=0 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.212+0000 7f3e4def8b80 1 client.4788 shutdown 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 msgr2=0x55dd7ad5fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 0x55dd7ad5fb30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f820 tx=0x7f3e3400fb30 comp rx=0 tx=0).stop 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 shutdown_connections 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad58b30 0x55dd7ad8ec20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad61120 0x55dd7ad5fb30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e1a7fc700 0 client.4788 ms_handle_reset on v2:172.21.15.83:6835/3706545143 2023-11-28T11:09:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad5e130 0x55dd7ad56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2045686996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad5f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 >> 172.21.15.83:0/2045686996 conn(0x55dd7ad33d60 msgr2=0x55dd7ad36010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 shutdown_connections 2023-11-28T11:09:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2045686996 wait complete. 2023-11-28T11:09:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.213+0000 7f3e4def8b80 20 client.4788 trim_cache size 0 max 16384 2023-11-28T11:09:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.230+0000 7f3e4def8b80 1 Processor -- start 2023-11-28T11:09:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.230+0000 7f3e4def8b80 1 -- start start 2023-11-28T11:09:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7ad31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad32100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad37a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad34410 con 0x55dd7ad5e130 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad7f5e0 con 0x55dd7ad60810 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad7fb10 con 0x55dd7ad61120 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7ad31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7ad31090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49276/0 (socket says 172.21.15.83:49276) 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/2553653800 learned_addr learned my addr 172.21.15.83:0/2553653800 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad37a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3affd700 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 msgr2=0x55dd7ad37a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad37a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 msgr2=0x55dd7ad32100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad32100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/2553653800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd7ac4f140 con 0x55dd7ad5e130 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad37a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7ad31090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e240583c0 tx=0x7f3e2406e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4204bac781d237 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e31ffb700 1 -- 172.21.15.83:0/2553653800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24071100 con 0x55dd7ad5e130 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e31ffb700 1 -- 172.21.15.83:0/2553653800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e24071260 con 0x55dd7ad5e130 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.231+0000 7f3e31ffb700 1 -- 172.21.15.83:0/2553653800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24061e70 con 0x55dd7ad5e130 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2553653800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 msgr2=0x55dd7ad31090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7ad31090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e240583c0 tx=0x7f3e2406e000 comp rx=0 tx=0).stop 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2553653800 shutdown_connections 2023-11-28T11:09:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad37a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7ad31090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/2553653800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad32100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2553653800 >> 172.21.15.83:0/2553653800 conn(0x55dd7ad33d60 msgr2=0x55dd7ad56330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2553653800 shutdown_connections 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.232+0000 7f3e4def8b80 1 -- 172.21.15.83:0/2553653800 wait complete. 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.233+0000 7f3e4def8b80 1 -- start start 2023-11-28T11:09:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.233+0000 7f3e4def8b80 1 Processor -- start 2023-11-28T11:09:01.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.233+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7add1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.233+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.233+0000 7f3e4def8b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad57c90 con 0x55dd7ad5e130 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7ad82820 con 0x55dd7ad60810 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd7add21d0 con 0x55dd7ad61120 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 10 client.0 _ll_register_callbacks cb 0x55dd7ab2e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7add1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7add1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49282/0 (socket says 172.21.15.83:49282) 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3892014116 learned_addr learned my addr 172.21.15.83:0/3892014116 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3bfff700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3affd700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 msgr2=0x55dd7ad580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 msgr2=0x55dd7ad822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 -- 172.21.15.83:0/3892014116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd7ab0a5e0 con 0x55dd7ad5e130 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e3b7fe700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7add1910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e24061bd0 tx=0x7f3e24020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a5916a7b45e024 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e1a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/3892014116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24071100 con 0x55dd7ad5e130 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/3892014116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e24061e80 con 0x55dd7ad5e130 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd7ac46160 con 0x55dd7ad5e130 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/3892014116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e24068120 con 0x55dd7ad5e130 2023-11-28T11:09:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 10 client.4794 fetch_fsmap learned FSMap version 9 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55dd7aadd200 con 0x55dd7ad5e130 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/3892014116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dd7ac46160 con 0x55dd7ad5e130 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.234+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/3892014116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3e24068280 con 0x55dd7ad5e130 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 10 client.4794 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 10 client.4794 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 20 client.4794 populate_metadata read hostname 'smithi083' 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad585f0 0x55dd7ad74ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 4 client.4794 mds_command: new command op to 4263 tid=0[{"prefix": "status"}] 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 --> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dd7ac30170 con 0x55dd7ad585f0 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e3affd700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad585f0 0x55dd7ad74ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e1a7fc700 10 client.4794 ms_handle_connect on v2:172.21.15.83:6835/3706545143 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e3affd700 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad585f0 0x55dd7ad74ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ba90aec9fb2ece server_cookie=62c26de68a7265ad in_seq=0 out_seq=0 2023-11-28T11:09:01.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e1a7fc700 1 -- 172.21.15.83:0/3892014116 <== mds.0 v2:172.21.15.83:6835/3706545143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55dd7ac30170 con 0x55dd7ad585f0 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e1a7fc700 10 client.4794 handle_command_reply: tid=0 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 client.4794 shutdown 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 msgr2=0x55dd7add1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7add1910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e24061bd0 tx=0x7f3e24020000 comp rx=0 tx=0).stop 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 shutdown_connections 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:6835/3706545143,v1:172.21.15.83:6836/3706545143] conn(0x55dd7ad585f0 0x55dd7ad74ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55dd7ad61120 0x55dd7ad580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e1a7fc700 0 client.4794 ms_handle_reset on v2:172.21.15.83:6835/3706545143 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd7ad5e130 0x55dd7add1910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 --2- 172.21.15.83:0/3892014116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd7ad60810 0x55dd7ad822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 >> 172.21.15.83:0/3892014116 conn(0x55dd7ad33d60 msgr2=0x55dd7ad378e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 shutdown_connections 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 1 -- 172.21.15.83:0/3892014116 wait complete. 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.235+0000 7f3e4def8b80 20 client.4794 trim_cache size 0 max 16384 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "69424f07-94c3-4dbc-8507-3551775e0a09", 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4263, 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-28T11:09:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-28T11:09:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 25.716435369999999, 2023-11-28T11:09:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 9, 2023-11-28T11:09:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 24, 2023-11-28T11:09:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T11:09:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 30.281826956 2023-11-28T11:09:01.247 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3e1c0682f0 msgr2=0x7f3e1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3e1c0682f0 0x7f3e1c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b0960 tx=0x7f3e2c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 msgr2=0x7f3e3c14ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 0x7f3e3c14ec50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e24000c00 tx=0x7f3e24004750 comp rx=0 tx=0).stop 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 shutdown_connections 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3e3c0af1f0 0x7f3e3c157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3e1c0682f0 0x7f3e1c06a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c06c370 0x7f3e3c14ec50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 --2- 172.21.15.83:0/4223087094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e3c0aee20 0x7f3e3c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 >> 172.21.15.83:0/4223087094 conn(0x7f3e3c0b2b60 msgr2=0x7f3e3c0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 shutdown_connections 2023-11-28T11:09:01.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.236+0000 7f3e41d6f700 1 -- 172.21.15.83:0/4223087094 wait complete. 2023-11-28T11:09:01.250 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T11:09:01.268 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T11:09:01.268 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T11:09:01.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.269 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:09:01.286 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-28T11:09:01.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T11:09:01.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-28T11:09:01.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.464 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:09:01.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-28T11:09:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2023-11-28T11:09:01.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:09:01.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.552 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2023-11-28T11:09:01.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2023-11-28T11:09:01.647 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2023-11-28T11:09:01.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:09:01.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.649 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2023-11-28T11:09:01.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:01.709 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2023-11-28T11:09:01.780 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2023-11-28T11:09:01.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:09:01.781 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-28T11:09:01.782 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:01.782 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:01.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:01.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.901+0000 7f15172a0700 1 Processor -- start 2023-11-28T11:09:01.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.902+0000 7f15172a0700 1 -- start start 2023-11-28T11:09:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.902+0000 7f15172a0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1510071710 0x7f15100cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.902+0000 7f15172a0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15101586b0 0x7f1510158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.902+0000 7f15172a0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100cac90 con 0x7f1510071ae0 2023-11-28T11:09:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.902+0000 7f15172a0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100ca3f0 con 0x7f15101586b0 2023-11-28T11:09:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.902+0000 7f15172a0700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100ca6d0 con 0x7f1510071710 2023-11-28T11:09:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151583d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15101586b0 0x7f1510158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151483b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1510071ae0 0x7f15100c98a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47150/0 (socket says 172.21.15.83:47150) 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151483b700 1 -- 172.21.15.83:0/3020182595 learned_addr learned my addr 172.21.15.83:0/3020182595 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151503c700 1 --2- 172.21.15.83:0/3020182595 >> v2:172.21.15.90:3300/0 conn(0x7f1510071710 0x7f15100cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151583d700 1 -- 172.21.15.83:0/3020182595 >> v2:172.21.15.90:3300/0 conn(0x7f1510071710 msgr2=0x7f15100cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151583d700 1 --2- 172.21.15.83:0/3020182595 >> v2:172.21.15.90:3300/0 conn(0x7f1510071710 0x7f15100cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151583d700 1 -- 172.21.15.83:0/3020182595 >> v1:172.21.15.83:6789/0 conn(0x7f1510071ae0 legacy=0x7f15100c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151583d700 1 -- 172.21.15.83:0/3020182595 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100ca9b0 con 0x7f15101586b0 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f151583d700 1 --2- 172.21.15.83:0/3020182595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f1510158aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1510072030 tx=0x7f150800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86846f1c2725c79e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f1507fff700 1 -- 172.21.15.83:0/3020182595 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800e070 con 0x7f15101586b0 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f1507fff700 1 -- 172.21.15.83:0/3020182595 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150800bbd0 con 0x7f15101586b0 2023-11-28T11:09:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.903+0000 7f1507fff700 1 -- 172.21.15.83:0/3020182595 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508012430 con 0x7f15101586b0 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 -- 172.21.15.83:0/3020182595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 msgr2=0x7f1510158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 --2- 172.21.15.83:0/3020182595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f1510158aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1510072030 tx=0x7f150800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 -- 172.21.15.83:0/3020182595 shutdown_connections 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 --2- 172.21.15.83:0/3020182595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f1510158aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 --2- 172.21.15.83:0/3020182595 >> v2:172.21.15.90:3300/0 conn(0x7f1510071710 0x7f15100cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 -- 172.21.15.83:0/3020182595 >> 172.21.15.83:0/3020182595 conn(0x7f15100cec90 msgr2=0x7f15100cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 -- 172.21.15.83:0/3020182595 shutdown_connections 2023-11-28T11:09:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.904+0000 7f15172a0700 1 -- 172.21.15.83:0/3020182595 wait complete. 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.905+0000 7f15172a0700 1 Processor -- start 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.905+0000 7f15172a0700 1 -- start start 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510071710 0x7f1510152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510071ae0 0x7f15100c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f15100c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100aeec0 con 0x7f15101586b0 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101527a0 con 0x7f1510071710 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100af110 con 0x7f1510071ae0 2023-11-28T11:09:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f15100c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f15100c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49300/0 (socket says 172.21.15.83:49300) 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 -- 172.21.15.83:0/3207415648 learned_addr learned my addr 172.21.15.83:0/3207415648 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151503c700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510071710 0x7f1510152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151483b700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510071ae0 0x7f15100c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 -- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510071ae0 msgr2=0x7f15100c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510071ae0 0x7f15100c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 -- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510071710 msgr2=0x7f1510152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510071710 0x7f1510152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 -- 172.21.15.83:0/3207415648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508009a70 con 0x7f15101586b0 2023-11-28T11:09:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f151583d700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f15100c8720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1508014040 tx=0x7f1508004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508012da0 con 0x7f15101586b0 2023-11-28T11:09:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508020070 con 0x7f15101586b0 2023-11-28T11:09:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508004280 con 0x7f15101586b0 2023-11-28T11:09:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.906+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1510158fe0 con 0x7f15101586b0 2023-11-28T11:09:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.907+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151006dae0 con 0x7f15101586b0 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.908+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15100c24a0 con 0x7f15101586b0 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.908+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f150802e070 con 0x7f15101586b0 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.908+0000 7f1505ffb700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.908+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1508022070 con 0x7f15101586b0 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.908+0000 7f151503c700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.909+0000 7f151503c700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f151006f080 tx=0x7f1500008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:01.913+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15080541c0 con 0x7f15101586b0 2023-11-28T11:09:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.138+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1510074320 con 0x7f15101586b0 2023-11-28T11:09:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.138+0000 7f1505ffb700 1 -- 172.21.15.83:0/3207415648 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f150802c070 con 0x7f15101586b0 2023-11-28T11:09:02.145 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 msgr2=0x7f14f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f151006f080 tx=0x7f1500008040 comp rx=0 tx=0).stop 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 msgr2=0x7f15100c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f15100c8720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1508014040 tx=0x7f1508004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 shutdown_connections 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15101586b0 0x7f15100c8720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510071710 0x7f1510152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 --2- 172.21.15.83:0/3207415648 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510071ae0 0x7f15100c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 >> 172.21.15.83:0/3207415648 conn(0x7f15100cec90 msgr2=0x7f15100cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 shutdown_connections 2023-11-28T11:09:02.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:02.140+0000 7f15172a0700 1 -- 172.21.15.83:0/3207415648 wait complete. 2023-11-28T11:09:02.153 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:02.154 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:02.155 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:02.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.155 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-28T11:09:02.172 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:02.172 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:02.172 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:02.172 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42331sec preferred_lft 42331sec 2023-11-28T11:09:02.173 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::22f1:9e4c:955b:b4a/64 scope link noprefixroute 2023-11-28T11:09:02.174 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:02.174 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:09:02.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.174 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:02.174 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:02.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:09:02.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2023-11-28T11:09:02.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2023-11-28T11:09:02.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:09:02.175 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:02.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.287 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:02.312 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:09:02.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.313 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-28T11:09:02.372 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-28T11:09:02.372 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:02.372 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:09:02.373 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:09:02.373 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:02.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.373 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:02.373 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:02.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:09:02.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:09:02.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:09:02.373 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:02.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.477 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:09:02.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.494 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2023-11-28T11:09:02.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.552 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:02.552 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:02.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:09:02.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T11:09:02.552 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:02.646 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T11:09:02.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.646 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:02.646 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:02.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T11:09:02.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:02.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T11:09:02.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T11:09:02.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T11:09:02.647 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:02.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:02.846 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:02.846 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:02.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-28T11:09:02.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:09:02.847 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:02.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:02.973 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:02.994 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:02.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:02.995 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:03.052 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-28T11:09:03.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:03.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:03.137 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:03.137 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:03.272 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:09:03.272 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:09:03.272 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:09:03.274 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:09:03.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:03.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:03.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:03.304 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:03.376 INFO:teuthology.orchestra.run.smithi083.stdout:client4806 2023-11-28T11:09:03.376 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:03.376 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:03.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:03.434 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:03.504 INFO:teuthology.orchestra.run.smithi083.stdout:client4806 2023-11-28T11:09:03.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:03.505 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:03.577 INFO:teuthology.orchestra.run.smithi083.stdout:client4806 2023-11-28T11:09:03.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:03.577 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/69424f07-94c3-4dbc-8507-3551775e0a09.client4806/status') 2023-11-28T11:09:03.657 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4806 (3)192.168.0.1:0/2090181240 2023-11-28T11:09:03.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2023-11-28T11:09:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2023-11-28T11:09:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2023-11-28T11:09:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:69 bytes copied, 2.2091e-05 s, 3.1 MB/s 2023-11-28T11:09:03.659 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-28T11:09:03.659 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:03.659 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:03.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:03.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.819+0000 7f0e04d27700 1 Processor -- start 2023-11-28T11:09:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.819+0000 7f0e04d27700 1 -- start start 2023-11-28T11:09:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0e04d27700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e00159830 0x7f0e000cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0e04d27700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e0006c5f0 0x7f0e0015de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0e04d27700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0006ca60 con 0x7f0e000b18a0 2023-11-28T11:09:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0e04d27700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0015d480 con 0x7f0e00159830 2023-11-28T11:09:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0e04d27700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0015e3d0 con 0x7f0e0006c5f0 2023-11-28T11:09:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0dfe59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e00159830 0x7f0e000cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0dfe59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e00159830 0x7f0e000cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49308/0 (socket says 172.21.15.83:49308) 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0dfe59c700 1 -- 172.21.15.83:0/4220330026 learned_addr learned my addr 172.21.15.83:0/4220330026 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0dfdd9b700 1 --2- 172.21.15.83:0/4220330026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e0006c5f0 0x7f0e0015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0dfe59c700 1 -- 172.21.15.83:0/4220330026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e0006c5f0 msgr2=0x7f0e0015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.820+0000 7f0dfe59c700 1 --2- 172.21.15.83:0/4220330026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e0006c5f0 0x7f0e0015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0dfe59c700 1 -- 172.21.15.83:0/4220330026 >> v1:172.21.15.90:6789/0 conn(0x7f0e000b18a0 legacy=0x7f0e000cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0dfe59c700 1 -- 172.21.15.83:0/4220330026 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000cf790 con 0x7f0e00159830 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0dfe59c700 1 --2- 172.21.15.83:0/4220330026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e00159830 0x7f0e000cf0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0df40098f0 tx=0x7f0df4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77867c5ade844627 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0dfd59a700 1 -- 172.21.15.83:0/4220330026 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4011830 con 0x7f0e00159830 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0dfd59a700 1 -- 172.21.15.83:0/4220330026 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df4011990 con 0x7f0e00159830 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0dfd59a700 1 -- 172.21.15.83:0/4220330026 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4011d20 con 0x7f0e00159830 2023-11-28T11:09:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0e04d27700 1 -- 172.21.15.83:0/4220330026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e00159830 msgr2=0x7f0e000cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0e04d27700 1 --2- 172.21.15.83:0/4220330026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e00159830 0x7f0e000cf0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0df40098f0 tx=0x7f0df4009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0e04d27700 1 -- 172.21.15.83:0/4220330026 shutdown_connections 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0e04d27700 1 --2- 172.21.15.83:0/4220330026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e00159830 0x7f0e000cf0d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0e04d27700 1 --2- 172.21.15.83:0/4220330026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e0006c5f0 0x7f0e0015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.821+0000 7f0e04d27700 1 -- 172.21.15.83:0/4220330026 >> 172.21.15.83:0/4220330026 conn(0x7f0e0006e9a0 msgr2=0x7f0e000cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.822+0000 7f0e04d27700 1 -- 172.21.15.83:0/4220330026 shutdown_connections 2023-11-28T11:09:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.822+0000 7f0e04d27700 1 -- 172.21.15.83:0/4220330026 wait complete. 2023-11-28T11:09:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.823+0000 7f0e04d27700 1 Processor -- start 2023-11-28T11:09:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.823+0000 7f0e04d27700 1 -- start start 2023-11-28T11:09:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 0x7f0e000c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e000b18a0 0x7f0e000cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e00159830 0x7f0e0015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0015a610 con 0x7f0e0006c5f0 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000c9b50 con 0x7f0e00159830 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0015a860 con 0x7f0e000b18a0 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 0x7f0e000c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 0x7f0e000c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49314/0 (socket says 172.21.15.83:49314) 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 -- 172.21.15.83:0/2142843139 learned_addr learned my addr 172.21.15.83:0/2142843139 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfdd9b700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e000b18a0 0x7f0e000cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfed9d700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e00159830 0x7f0e0015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 -- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e000b18a0 msgr2=0x7f0e000cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e000b18a0 0x7f0e000cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 -- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e00159830 msgr2=0x7f0e0015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e00159830 0x7f0e0015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 -- 172.21.15.83:0/2142843139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4009550 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0dfe59c700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 0x7f0e000c9200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009c40 tx=0x7f0df4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df40036a0 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df401e070 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0015d690 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4003a40 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.824+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e000cfa70 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.825+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0015d970 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.826+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0df400f050 con 0x7f0e0006c5f0 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.826+0000 7f0deeffd700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0de00682f0 0x7f0de006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.826+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0df4086f80 con 0x7f0e0006c5f0 2023-11-28T11:09:03.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.826+0000 7f0dfdd9b700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0de00682f0 0x7f0de006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.826+0000 7f0dfdd9b700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0de00682f0 0x7f0de006a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e000cb970 tx=0x7f0df0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:03.830+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0df40534f0 con 0x7f0e0006c5f0 2023-11-28T11:09:04.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.057+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e0015b190 con 0x7f0e0006c5f0 2023-11-28T11:09:04.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.058+0000 7f0deeffd700 1 -- 172.21.15.83:0/2142843139 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0df4020020 con 0x7f0e0006c5f0 2023-11-28T11:09:04.059 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0de00682f0 msgr2=0x7f0de006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0de00682f0 0x7f0de006a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e000cb970 tx=0x7f0df0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 msgr2=0x7f0e000c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 0x7f0e000c9200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009c40 tx=0x7f0df4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 shutdown_connections 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e00159830 0x7f0e0015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0de00682f0 0x7f0de006a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0006c5f0 0x7f0e000c9200 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 --2- 172.21.15.83:0/2142843139 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e000b18a0 0x7f0e000cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 >> 172.21.15.83:0/2142843139 conn(0x7f0e0006e9a0 msgr2=0x7f0e000cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 shutdown_connections 2023-11-28T11:09:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:04.060+0000 7f0e04d27700 1 -- 172.21.15.83:0/2142843139 wait complete. 2023-11-28T11:09:04.073 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:04.074 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:09:04.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.074 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-28T11:09:04.092 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42329sec preferred_lft 42329sec 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::22f1:9e4c:955b:b4a/64 scope link noprefixroute 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::80fe:9fff:fe99:6cb2/64 scope link 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:09:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::1c9b:f5ff:fe74:5abe/64 scope link tentative 2023-11-28T11:09:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.095 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:04.158 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:09:04.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.160 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-28T11:09:04.219 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-28T11:09:04.219 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:04.219 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:09:04.219 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:09:04.219 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:04.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.220 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:04.220 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:04.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:09:04.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:09:04.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:09:04.220 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:04.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.324 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:09:04.341 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:09:04.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.342 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2023-11-28T11:09:04.398 INFO:teuthology.orchestra.run.smithi083.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T11:09:04.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.399 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:04.399 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:04.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:09:04.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T11:09:04.399 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:04.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 72:f9:1b:f2:2d:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::70f9:1bff:fef2:2d44/64 scope link tentative 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:09:04.625 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 72:f9:1b:f2:2d:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::70f9:1bff:fef2:2d44/64 scope link tentative 2023-11-28T11:09:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:04.635 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T11:09:04.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T11:09:04.636 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:04.894 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:04.894 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:04.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.1 up 2023-11-28T11:09:04.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:09:04.894 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:05.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:05.004 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:05.028 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:09:05.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:05.028 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:05.084 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-28T11:09:05.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:05.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:05.170 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:05.171 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:05.199 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 2@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:05.201 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:05.274 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:05.274 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:05.274 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:05.274 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 2@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:09:05.275 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:09:05.276 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:09:05.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:05.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:05.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:05.343 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:client4589 2023-11-28T11:09:05.415 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:05.416 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:05.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:05.473 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:client4589 2023-11-28T11:09:05.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:05.545 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:client4589 2023-11-28T11:09:05.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:05.617 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/69424f07-94c3-4dbc-8507-3551775e0a09.client4589/status') 2023-11-28T11:09:05.697 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4589 (3)172.21.15.83:0/3761018214 2023-11-28T11:09:05.697 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2023-11-28T11:09:05.698 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2023-11-28T11:09:05.698 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2023-11-28T11:09:05.698 INFO:teuthology.orchestra.run.smithi083.stderr:70 bytes copied, 2.2113e-05 s, 3.2 MB/s 2023-11-28T11:09:05.699 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-28T11:09:05.699 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:05.699 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:05.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:05.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.860+0000 7f203453c700 1 Processor -- start 2023-11-28T11:09:05.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.860+0000 7f203453c700 1 -- start start 2023-11-28T11:09:05.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f203453c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f202c159830 0x7f202c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f203453c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c06ca60 con 0x7f202c06c5f0 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f203453c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c0cf790 con 0x7f202c0b18a0 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f203453c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c15d480 con 0x7f202c159830 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f2031ad7700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f202c06c5f0 0x7f202c15de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47188/0 (socket says 172.21.15.83:47188) 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f2031ad7700 1 -- 172.21.15.83:0/2824254827 learned_addr learned my addr 172.21.15.83:0/2824254827 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293449898 0 0) 0x7f202c06ca60 con 0x7f202c06c5f0 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20180035b0 con 0x7f202c06c5f0 2023-11-28T11:09:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20322d8700 1 --2- 172.21.15.83:0/2824254827 >> v2:172.21.15.90:3300/0 conn(0x7f202c159830 0x7f202c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2704481174 0 0) 0x7f20180035b0 con 0x7f202c06c5f0 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 >> v2:172.21.15.90:3300/0 conn(0x7f202c159830 msgr2=0x7f202c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 --2- 172.21.15.83:0/2824254827 >> v2:172.21.15.90:3300/0 conn(0x7f202c159830 0x7f202c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 >> v1:172.21.15.90:6789/0 conn(0x7f202c0b18a0 legacy=0x7f202c0cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0cfa70 con 0x7f202c06c5f0 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f201c001670 con 0x7f202c06c5f0 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 >> v1:172.21.15.83:6789/0 conn(0x7f202c06c5f0 legacy=0x7f202c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.861+0000 7f20312d6700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2018003a00 0x7f2018003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2018004850 0x7f2018004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018005200 0x7f201800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c15d480 con 0x7f2018003a00 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201800c080 con 0x7f2018004850 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201800c240 con 0x7f2018005200 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2018003a00 0x7f2018003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f2031ad7700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2018004850 0x7f2018004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f2032ad9700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018005200 0x7f201800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 -- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018005200 msgr2=0x7f201800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018005200 0x7f201800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 -- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2018004850 msgr2=0x7f2018004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2018004850 0x7f2018004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 -- 172.21.15.83:0/2824254827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180035b0 con 0x7f2018003a00 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f2032ad9700 1 -- 172.21.15.83:0/2824254827 reap_dead start 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20322d8700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2018003a00 0x7f2018003e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2028009b60 tx=0x7f2018005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e574e401272a30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800e6c0 con 0x7f2018003a00 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800e820 con 0x7f2018003a00 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.862+0000 7f20312d6700 1 -- 172.21.15.83:0/2824254827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800ebb0 con 0x7f2018003a00 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 -- 172.21.15.83:0/2824254827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2018003a00 msgr2=0x7f2018003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2018003a00 0x7f2018003e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2028009b60 tx=0x7f2018005880 comp rx=0 tx=0).stop 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 -- 172.21.15.83:0/2824254827 shutdown_connections 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 --2- 172.21.15.83:0/2824254827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2018003a00 0x7f2018003e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 -- 172.21.15.83:0/2824254827 >> 172.21.15.83:0/2824254827 conn(0x7f202c06e9a0 msgr2=0x7f202c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 -- 172.21.15.83:0/2824254827 shutdown_connections 2023-11-28T11:09:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.863+0000 7f203453c700 1 -- 172.21.15.83:0/2824254827 wait complete. 2023-11-28T11:09:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.864+0000 7f203453c700 1 Processor -- start 2023-11-28T11:09:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.864+0000 7f203453c700 1 -- start start 2023-11-28T11:09:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018003a00 0x7f202c151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 0x7f202c0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c0b0650 0x7f202c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0ac090 con 0x7f202c15c8d0 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c15cd40 con 0x7f202c0b0650 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0abdf0 con 0x7f2018003a00 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 0x7f202c0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 0x7f202c0ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49340/0 (socket says 172.21.15.83:49340) 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 -- 172.21.15.83:0/10742116 learned_addr learned my addr 172.21.15.83:0/10742116 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f20322d8700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018003a00 0x7f202c151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2032ad9700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c0b0650 0x7f202c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 -- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018003a00 msgr2=0x7f202c151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018003a00 0x7f202c151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 -- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c0b0650 msgr2=0x7f202c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c0b0650 0x7f202c0b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 -- 172.21.15.83:0/10742116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c15d970 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2031ad7700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 0x7f202c0ab8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f201c0072c0 tx=0x7f201c0075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00f600 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c00f7a0 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c15d690 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00fb50 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.865+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c159c00 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.866+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c0d0ce0 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.867+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f201c00fcb0 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.867+0000 7f2022ffd700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f20100682f0 0x7f201006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.867+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f201c085f30 con 0x7f202c15c8d0 2023-11-28T11:09:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.867+0000 7f20322d8700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f20100682f0 0x7f201006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.868+0000 7f20322d8700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f20100682f0 0x7f201006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20280090b0 tx=0x7f202801e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:05.871+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f201c0524a0 con 0x7f202c15c8d0 2023-11-28T11:09:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.097+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f202c159020 con 0x7f202c15c8d0 2023-11-28T11:09:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.097+0000 7f2022ffd700 1 -- 172.21.15.83:0/10742116 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f201c055ac0 con 0x7f202c15c8d0 2023-11-28T11:09:06.099 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f20100682f0 msgr2=0x7f201006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f20100682f0 0x7f201006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20280090b0 tx=0x7f202801e040 comp rx=0 tx=0).stop 2023-11-28T11:09:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 msgr2=0x7f202c0ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 0x7f202c0ab8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f201c0072c0 tx=0x7f201c0075d0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 shutdown_connections 2023-11-28T11:09:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c0b0650 0x7f202c0b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2018003a00 0x7f202c151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f20100682f0 0x7f201006a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 --2- 172.21.15.83:0/10742116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f202c15c8d0 0x7f202c0ab8b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 >> 172.21.15.83:0/10742116 conn(0x7f202c06e9a0 msgr2=0x7f202c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 shutdown_connections 2023-11-28T11:09:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:06.099+0000 7f203453c700 1 -- 172.21.15.83:0/10742116 wait complete. 2023-11-28T11:09:06.112 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:06.113 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:09:06.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.114 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2023-11-28T11:09:06.136 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:06.137 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:06.137 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:06.138 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:06.138 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:06.138 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:06.138 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42375sec preferred_lft 42375sec 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::3743:adc3:3156:5865/64 scope link noprefixroute 2023-11-28T11:09:06.141 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:06.142 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:09:06.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.142 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:06.142 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:06.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:09:06.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2023-11-28T11:09:06.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2023-11-28T11:09:06.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:09:06.144 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:06.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.320 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:06.348 INFO:teuthology.orchestra.run.smithi090.stdout:1 2023-11-28T11:09:06.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.350 DEBUG:teuthology.orchestra.run.smithi090:> route 2023-11-28T11:09:06.442 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2023-11-28T11:09:06.442 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:06.442 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:09:06.442 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:09:06.442 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:06.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.443 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:06.443 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:06.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:09:06.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:09:06.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:09:06.443 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:06.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.648 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2023-11-28T11:09:06.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.665 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2023-11-28T11:09:06.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.722 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:06.722 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:06.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:09:06.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T11:09:06.722 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:06.814 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T11:09:06.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T11:09:06.815 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:07.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.025 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:07.026 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:07.026 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2023-11-28T11:09:07.026 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:09:07.026 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:07.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:07.125 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:07.146 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:09:07.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:07.147 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:07.203 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-28T11:09:07.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:07.297 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:07.297 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with new device syntax: 1@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:07.431 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:07.459 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with new device syntax: 1@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:09:07.460 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:09:07.469 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:09:07.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:07.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.501 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:07.573 INFO:teuthology.orchestra.run.smithi090.stdout:client4717 2023-11-28T11:09:07.573 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:07.573 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:07.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.631 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:client4717 2023-11-28T11:09:07.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.704 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:07.775 INFO:teuthology.orchestra.run.smithi090.stdout:client4717 2023-11-28T11:09:07.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:07.776 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/69424f07-94c3-4dbc-8507-3551775e0a09.client4717/status') 2023-11-28T11:09:07.856 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.4717 (3)192.168.0.1:0/2244243331 2023-11-28T11:09:07.856 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2023-11-28T11:09:07.856 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2023-11-28T11:09:07.857 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2023-11-28T11:09:07.857 INFO:teuthology.orchestra.run.smithi090.stderr:69 bytes copied, 2.2738e-05 s, 3.0 MB/s 2023-11-28T11:09:07.857 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-28T11:09:07.858 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:07.858 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:07.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:07.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.990+0000 7fb3a1e74700 1 Processor -- start 2023-11-28T11:09:07.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.990+0000 7fb3a1e74700 1 -- start start 2023-11-28T11:09:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb3a1e74700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c159830 0x7fb39c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb3a1e74700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb39c06c5f0 0x7fb39c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb3a1e74700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c06ca60 con 0x7fb39c0b18a0 2023-11-28T11:09:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb3a1e74700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c15d480 con 0x7fb39c06c5f0 2023-11-28T11:09:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb3a1e74700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c15e3d0 con 0x7fb39c159830 2023-11-28T11:09:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb39bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb39c0b18a0 0x7fb39c0cd9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47212/0 (socket says 172.21.15.83:47212) 2023-11-28T11:09:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb39bfff700 1 -- 172.21.15.83:0/4226275252 learned_addr learned my addr 172.21.15.83:0/4226275252 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb39affd700 1 --2- 172.21.15.83:0/4226275252 >> v2:172.21.15.90:3300/0 conn(0x7fb39c06c5f0 0x7fb39c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.991+0000 7fb39b7fe700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c159830 0x7fb39c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39a7fc700 1 -- 172.21.15.83:0/4226275252 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246133484 0 0) 0x7fb39c06ca60 con 0x7fb39c0b18a0 2023-11-28T11:09:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39a7fc700 1 -- 172.21.15.83:0/4226275252 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3840035b0 con 0x7fb39c0b18a0 2023-11-28T11:09:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39affd700 1 -- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c159830 msgr2=0x7fb39c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39affd700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c159830 0x7fb39c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39affd700 1 -- 172.21.15.83:0/4226275252 >> v1:172.21.15.83:6789/0 conn(0x7fb39c0b18a0 legacy=0x7fb39c0cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39affd700 1 -- 172.21.15.83:0/4226275252 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0cf790 con 0x7fb39c06c5f0 2023-11-28T11:09:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39b7fe700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c159830 0x7fb39c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39affd700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c15de90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009840 tx=0x7fb38c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975dfbefd9190a2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39a7fc700 1 -- 172.21.15.83:0/4226275252 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c011830 con 0x7fb39c06c5f0 2023-11-28T11:09:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39a7fc700 1 -- 172.21.15.83:0/4226275252 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c011990 con 0x7fb39c06c5f0 2023-11-28T11:09:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.992+0000 7fb39a7fc700 1 -- 172.21.15.83:0/4226275252 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c011d20 con 0x7fb39c06c5f0 2023-11-28T11:09:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 -- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 msgr2=0x7fb39c15de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c15de90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009840 tx=0x7fb38c009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:07.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 -- 172.21.15.83:0/4226275252 shutdown_connections 2023-11-28T11:09:07.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c159830 0x7fb39c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/4226275252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c15de90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 -- 172.21.15.83:0/4226275252 >> 172.21.15.83:0/4226275252 conn(0x7fb39c06e9a0 msgr2=0x7fb39c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 -- 172.21.15.83:0/4226275252 shutdown_connections 2023-11-28T11:09:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.993+0000 7fb3a1e74700 1 -- 172.21.15.83:0/4226275252 wait complete. 2023-11-28T11:09:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.994+0000 7fb3a1e74700 1 Processor -- start 2023-11-28T11:09:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.994+0000 7fb3a1e74700 1 -- start start 2023-11-28T11:09:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c0b18a0 0x7fb39c0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 0x7fb39c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1f1b40 con 0x7fb39c159830 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c154840 con 0x7fb39c06c5f0 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0c7e40 con 0x7fb39c0b18a0 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 0x7fb39c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 0x7fb39c06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49358/0 (socket says 172.21.15.83:49358) 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 -- 172.21.15.83:0/918076466 learned_addr learned my addr 172.21.15.83:0/918076466 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39affd700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c0b18a0 0x7fb39c0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39b7fe700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 -- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c0b18a0 msgr2=0x7fb39c0c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c0b18a0 0x7fb39c0c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 -- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 msgr2=0x7fb39c154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 -- 172.21.15.83:0/918076466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900094a0 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb39bfff700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 0x7fb39c06ab00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb39400caa0 tx=0x7fb39400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39400aa80 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39400abe0 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0094a0 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.995+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394012400 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.996+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c15d690 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.997+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39c0cfa70 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.997+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb394019030 con 0x7fb39c159830 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.997+0000 7fb383fff700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb3880682f0 0x7fb38806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.997+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb394085bb0 con 0x7fb39c159830 2023-11-28T11:09:08.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.997+0000 7fb39b7fe700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb3880682f0 0x7fb38806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:07.998+0000 7fb39b7fe700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb3880682f0 0x7fb38806a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0ccb40 tx=0x7fb390008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.002+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb39408c080 con 0x7fb39c159830 2023-11-28T11:09:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.227+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb39c15d970 con 0x7fb39c159830 2023-11-28T11:09:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.228+0000 7fb383fff700 1 -- 172.21.15.83:0/918076466 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb394017070 con 0x7fb39c159830 2023-11-28T11:09:08.229 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb3880682f0 msgr2=0x7fb38806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb3880682f0 0x7fb38806a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0ccb40 tx=0x7fb390008040 comp rx=0 tx=0).stop 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 msgr2=0x7fb39c06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 0x7fb39c06ab00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb39400caa0 tx=0x7fb39400ce60 comp rx=0 tx=0).stop 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 shutdown_connections 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb3880682f0 0x7fb38806a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb39c159830 0x7fb39c06ab00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb39c06c5f0 0x7fb39c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 --2- 172.21.15.83:0/918076466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb39c0b18a0 0x7fb39c0c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 >> 172.21.15.83:0/918076466 conn(0x7fb39c06e9a0 msgr2=0x7fb39c0cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 shutdown_connections 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:08.230+0000 7fb3a1e74700 1 -- 172.21.15.83:0/918076466 wait complete. 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:08.244 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:08.245 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:09:08.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.245 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42373sec preferred_lft 42373sec 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::3743:adc3:3156:5865/64 scope link noprefixroute 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0a:be:6c:95:37:62 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::834:c9ff:feff:e8bb/64 scope link 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi090.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0a:be:6c:95:37:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::8be:6cff:fe95:3762/64 scope link 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.267 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:08.330 INFO:teuthology.orchestra.run.smithi090.stdout:1 2023-11-28T11:09:08.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.332 DEBUG:teuthology.orchestra.run.smithi090:> route 2023-11-28T11:09:08.391 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2023-11-28T11:09:08.391 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:08.391 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:09:08.391 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:09:08.391 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:08.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.392 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:08.392 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:08.392 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:09:08.392 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:09:08.392 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:09:08.392 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:08.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.496 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2023-11-28T11:09:08.512 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:09:08.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.513 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2023-11-28T11:09:08.569 INFO:teuthology.orchestra.run.smithi090.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T11:09:08.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.569 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:08.570 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:08.570 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:09:08.570 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T11:09:08.570 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:08.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.663 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:09:08.702 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:08.702 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:08.703 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:08.703 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.703 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:08.703 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.703 INFO:teuthology.orchestra.run.smithi090.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:08.703 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether b6:6b:15:9e:4f:87 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:08.704 INFO:teuthology.orchestra.run.smithi090.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:08.704 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.704 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::b46b:15ff:fe9e:4f87/64 scope link 2023-11-28T11:09:08.704 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.711 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:09:08.787 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:08.788 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:08.788 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:08.788 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.788 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:08.788 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.789 INFO:teuthology.orchestra.run.smithi090.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:08.789 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether b6:6b:15:9e:4f:87 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:08.789 INFO:teuthology.orchestra.run.smithi090.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:08.789 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.789 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::b46b:15ff:fe9e:4f87/64 scope link 2023-11-28T11:09:08.789 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:08.800 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T11:09:08.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:08.800 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:08.800 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:08.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T11:09:08.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:08.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T11:09:08.801 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T11:09:08.801 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T11:09:08.801 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:09.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.053 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2023-11-28T11:09:09.053 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2023-11-28T11:09:09.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.1 up 2023-11-28T11:09:09.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:09:09.054 DEBUG:teuthology.orchestra.run.smithi090:> ') 2023-11-28T11:09:09.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:09.094 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:09.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:09.160 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:09.182 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:09:09.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:09.183 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:09.238 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-28T11:09:09.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:09.324 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:09.324 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:09.347 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with new device syntax: 3@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:09.348 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with new device syntax: 3@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:09:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:09:09.417 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:09:09.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.418 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:09.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.444 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:09.516 INFO:teuthology.orchestra.run.smithi090.stdout:client4827 2023-11-28T11:09:09.516 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:09.516 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:09.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.574 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:09.645 INFO:teuthology.orchestra.run.smithi090.stdout:client4827 2023-11-28T11:09:09.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.646 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:09.717 INFO:teuthology.orchestra.run.smithi090.stdout:client4827 2023-11-28T11:09:09.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:09.718 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/69424f07-94c3-4dbc-8507-3551775e0a09.client4827/status') 2023-11-28T11:09:09.797 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.4827 (3)172.21.15.90:0/257668644 2023-11-28T11:09:09.797 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2023-11-28T11:09:09.797 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2023-11-28T11:09:09.797 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2023-11-28T11:09:09.797 INFO:teuthology.orchestra.run.smithi090.stderr:69 bytes copied, 2.1151e-05 s, 3.3 MB/s 2023-11-28T11:09:09.798 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T11:09:09.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.949+0000 7fb62c28c700 1 Processor -- start 2023-11-28T11:09:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.949+0000 7fb62c28c700 1 -- start start 2023-11-28T11:09:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62c28c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb62415c850 0x7fb6240ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62c28c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb62415cc20 0x7fb6240ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62c28c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb624161450 con 0x7fb6240d09f0 2023-11-28T11:09:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62c28c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6240ce630 con 0x7fb62415cc20 2023-11-28T11:09:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62c28c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6241608d0 con 0x7fb62415c850 2023-11-28T11:09:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb629827700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb62415cc20 0x7fb6240ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62a829700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb6240d09f0 0x7fb62406f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47226/0 (socket says 172.21.15.83:47226) 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb629827700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb62415cc20 0x7fb6240ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49364/0 (socket says 172.21.15.83:49364) 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62a829700 1 -- 172.21.15.83:0/456588399 learned_addr learned my addr 172.21.15.83:0/456588399 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.950+0000 7fb62a028700 1 --2- 172.21.15.83:0/456588399 >> v2:172.21.15.90:3300/0 conn(0x7fb62415c850 0x7fb6240ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629827700 1 -- 172.21.15.83:0/456588399 >> v2:172.21.15.90:3300/0 conn(0x7fb62415c850 msgr2=0x7fb6240ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629827700 1 --2- 172.21.15.83:0/456588399 >> v2:172.21.15.90:3300/0 conn(0x7fb62415c850 0x7fb6240ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629827700 1 -- 172.21.15.83:0/456588399 >> v1:172.21.15.83:6789/0 conn(0x7fb6240d09f0 legacy=0x7fb62406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629827700 1 -- 172.21.15.83:0/456588399 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624160bb0 con 0x7fb62415cc20 2023-11-28T11:09:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629026700 1 -- 172.21.15.83:0/456588399 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176846663 0 0) 0x7fb624161450 con 0x7fb6240d09f0 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629827700 1 --2- 172.21.15.83:0/456588399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb62415cc20 0x7fb6240ceff0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb62406ecb0 tx=0x7fb614009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10e7df518a78aa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629026700 1 -- 172.21.15.83:0/456588399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011830 con 0x7fb62415cc20 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629026700 1 -- 172.21.15.83:0/456588399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614011990 con 0x7fb62415cc20 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb629026700 1 -- 172.21.15.83:0/456588399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011d20 con 0x7fb62415cc20 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb62c28c700 1 -- 172.21.15.83:0/456588399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb62415cc20 msgr2=0x7fb6240ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.951+0000 7fb62c28c700 1 --2- 172.21.15.83:0/456588399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb62415cc20 0x7fb6240ceff0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb62406ecb0 tx=0x7fb614009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.952+0000 7fb62c28c700 1 -- 172.21.15.83:0/456588399 shutdown_connections 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.952+0000 7fb62c28c700 1 --2- 172.21.15.83:0/456588399 >> v2:172.21.15.90:3300/0 conn(0x7fb62415c850 0x7fb6240ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.952+0000 7fb62c28c700 1 --2- 172.21.15.83:0/456588399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb62415cc20 0x7fb6240ceff0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.952+0000 7fb62c28c700 1 -- 172.21.15.83:0/456588399 >> 172.21.15.83:0/456588399 conn(0x7fb6240cd960 msgr2=0x7fb6240cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.952+0000 7fb62c28c700 1 -- 172.21.15.83:0/456588399 shutdown_connections 2023-11-28T11:09:09.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.952+0000 7fb62c28c700 1 -- 172.21.15.83:0/456588399 wait complete. 2023-11-28T11:09:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.953+0000 7fb62c28c700 1 Processor -- start 2023-11-28T11:09:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.953+0000 7fb62c28c700 1 -- start start 2023-11-28T11:09:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 0x7fb6240c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb62415c850 0x7fb62415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb62415cc20 0x7fb62415f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240c5b60 con 0x7fb6240d09f0 2023-11-28T11:09:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624160320 con 0x7fb62415c850 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62415fa80 con 0x7fb62415cc20 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 0x7fb6240c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 0x7fb6240c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49368/0 (socket says 172.21.15.83:49368) 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 -- 172.21.15.83:0/3347675272 learned_addr learned my addr 172.21.15.83:0/3347675272 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb629827700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb62415c850 0x7fb62415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a829700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb62415cc20 0x7fb62415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 -- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb62415cc20 msgr2=0x7fb62415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb62415cc20 0x7fb62415f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 -- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb62415c850 msgr2=0x7fb62415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb62415c850 0x7fb62415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 -- 172.21.15.83:0/3347675272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140094a0 con 0x7fb6240d09f0 2023-11-28T11:09:09.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62a028700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 0x7fb6240c4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9b0 tx=0x7fb62000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620009710 con 0x7fb6240d09f0 2023-11-28T11:09:09.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62000ee50 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb624161170 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62000ca50 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.954+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb624160e90 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.955+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb624160560 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.956+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb620019030 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.956+0000 7fb61affd700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb6080682f0 0x7fb60806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.956+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb620085e00 con 0x7fb6240d09f0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.956+0000 7fb629827700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb6080682f0 0x7fb60806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.957+0000 7fb629827700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb6080682f0 0x7fb60806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb6140090b0 tx=0x7fb61401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:09.961+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb620052370 con 0x7fb6240d09f0 2023-11-28T11:09:10.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.143+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb624074320 con 0x7fb6240d09f0 2023-11-28T11:09:10.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.145+0000 7fb61affd700 1 -- 172.21.15.83:0/3347675272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fb620055990 con 0x7fb6240d09f0 2023-11-28T11:09:10.146 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:10.147 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb6080682f0 msgr2=0x7fb60806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb6080682f0 0x7fb60806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb6140090b0 tx=0x7fb61401c040 comp rx=0 tx=0).stop 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 msgr2=0x7fb6240c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 0x7fb6240c4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9b0 tx=0x7fb62000ecc0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 shutdown_connections 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb62415cc20 0x7fb62415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb6080682f0 0x7fb60806a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6240d09f0 0x7fb6240c4060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 --2- 172.21.15.83:0/3347675272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb62415c850 0x7fb62415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 >> 172.21.15.83:0/3347675272 conn(0x7fb6240cd960 msgr2=0x7fb62406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 shutdown_connections 2023-11-28T11:09:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.147+0000 7fb62c28c700 1 -- 172.21.15.83:0/3347675272 wait complete. 2023-11-28T11:09:10.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:10.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.287+0000 7f3808fcd700 1 Processor -- start 2023-11-28T11:09:10.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.287+0000 7f3808fcd700 1 -- start start 2023-11-28T11:09:10.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f3808fcd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38040aee20 0x7f38040b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f3808fcd700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38040b5b00 con 0x7f380406c370 2023-11-28T11:09:10.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f3808fcd700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380406c800 con 0x7f38040af1f0 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f3808fcd700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380406cae0 con 0x7f38040aee20 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f380259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38040aee20 0x7f38040b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f3801d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f380406c370 0x7f380406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47238/0 (socket says 172.21.15.83:47238) 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.288+0000 7f3801d9b700 1 -- 172.21.15.83:0/1566231893 learned_addr learned my addr 172.21.15.83:0/1566231893 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380259c700 1 -- 172.21.15.83:0/1566231893 >> v1:172.21.15.90:6789/0 conn(0x7f38040af1f0 legacy=0x7f38040cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380259c700 1 -- 172.21.15.83:0/1566231893 >> v1:172.21.15.83:6789/0 conn(0x7f380406c370 legacy=0x7f380406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380259c700 1 -- 172.21.15.83:0/1566231893 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380406cdc0 con 0x7f38040aee20 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380159a700 1 -- 172.21.15.83:0/1566231893 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271414519 0 0) 0x7f37f4001110 con 0x7f380406c370 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380259c700 1 --2- 172.21.15.83:0/1566231893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040aee20 0x7f38040b55c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009820 tx=0x7f37f8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1891616f80426a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380159a700 1 -- 172.21.15.83:0/1566231893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8005e30 con 0x7f38040aee20 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380159a700 1 -- 172.21.15.83:0/1566231893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8006030 con 0x7f38040aee20 2023-11-28T11:09:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f380159a700 1 -- 172.21.15.83:0/1566231893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80063e0 con 0x7f38040aee20 2023-11-28T11:09:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f3808fcd700 1 -- 172.21.15.83:0/1566231893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040aee20 msgr2=0x7f38040b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.289+0000 7f3808fcd700 1 --2- 172.21.15.83:0/1566231893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040aee20 0x7f38040b55c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009820 tx=0x7f37f8009b30 comp rx=0 tx=0).stop 2023-11-28T11:09:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.290+0000 7f3808fcd700 1 -- 172.21.15.83:0/1566231893 shutdown_connections 2023-11-28T11:09:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.290+0000 7f3808fcd700 1 --2- 172.21.15.83:0/1566231893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040aee20 0x7f38040b55c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.290+0000 7f3808fcd700 1 -- 172.21.15.83:0/1566231893 >> 172.21.15.83:0/1566231893 conn(0x7f38040b2b60 msgr2=0x7f38040b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.290+0000 7f3808fcd700 1 -- 172.21.15.83:0/1566231893 shutdown_connections 2023-11-28T11:09:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.290+0000 7f3808fcd700 1 -- 172.21.15.83:0/1566231893 wait complete. 2023-11-28T11:09:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.291+0000 7f3808fcd700 1 Processor -- start 2023-11-28T11:09:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.291+0000 7f3808fcd700 1 -- start start 2023-11-28T11:09:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f380406c370 0x7f3804157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38040aee20 0x7f380409d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 0x7f380415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38041587a0 con 0x7f38040af1f0 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804159e20 con 0x7f38040aee20 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380415a100 con 0x7f380406c370 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 0x7f380415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 0x7f380415aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49386/0 (socket says 172.21.15.83:49386) 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 -- 172.21.15.83:0/972315852 learned_addr learned my addr 172.21.15.83:0/972315852 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3801d9b700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38040aee20 0x7f380409d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f380259c700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f380406c370 0x7f3804157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 -- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f380406c370 msgr2=0x7f3804157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f380406c370 0x7f3804157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 -- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38040aee20 msgr2=0x7f380409d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38040aee20 0x7f380409d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 -- 172.21.15.83:0/972315852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009430 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f380259c700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f380406c370 0x7f3804157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3802d9d700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 0x7f380415aec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00cb50 tx=0x7f37fc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc003c00 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc003d60 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380406fc70 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00ab70 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.292+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380406d0a0 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.293+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380406d380 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.294+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f37fc010050 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.294+0000 7f37f2ffd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f37e40682f0 0x7f37e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.294+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f37fc085e60 con 0x7f38040af1f0 2023-11-28T11:09:10.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.294+0000 7f380259c700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f37e40682f0 0x7f37e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.294+0000 7f380259c700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f37e40682f0 0x7f37e406a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f37f8013040 tx=0x7f37f8008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.298+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f37fc0523d0 con 0x7f38040af1f0 2023-11-28T11:09:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.480+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38040742b0 con 0x7f38040af1f0 2023-11-28T11:09:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.480+0000 7f37f2ffd700 1 -- 172.21.15.83:0/972315852 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f37fc0559f0 con 0x7f38040af1f0 2023-11-28T11:09:10.482 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f37e40682f0 msgr2=0x7f37e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f37e40682f0 0x7f37e406a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f37f8013040 tx=0x7f37f8008070 comp rx=0 tx=0).stop 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 msgr2=0x7f380415aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 0x7f380415aec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00cb50 tx=0x7f37fc00cf10 comp rx=0 tx=0).stop 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 shutdown_connections 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f37e40682f0 0x7f37e406a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38040af1f0 0x7f380415aec0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f380406c370 0x7f3804157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 --2- 172.21.15.83:0/972315852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38040aee20 0x7f380409d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 >> 172.21.15.83:0/972315852 conn(0x7f38040b2b60 msgr2=0x7f380406ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 shutdown_connections 2023-11-28T11:09:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.482+0000 7f3808fcd700 1 -- 172.21.15.83:0/972315852 wait complete. 2023-11-28T11:09:10.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.618+0000 7f2733e68700 1 Processor -- start 2023-11-28T11:09:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.618+0000 7f2733e68700 1 -- start start 2023-11-28T11:09:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2733e68700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f272c0b1900 0x7f272c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2733e68700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f272c0b5600 0x7f272c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2733e68700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c06bc10 con 0x7f272c15e000 2023-11-28T11:09:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2733e68700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c06cc20 con 0x7f272c0b5600 2023-11-28T11:09:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2733e68700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c06be60 con 0x7f272c0b1900 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731403700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f272c0b5600 0x7f272c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2732405700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f272c15e000 0x7f272c0cb870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:45600/0 (socket says 172.21.15.83:45600) 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2732405700 1 -- 172.21.15.83:0/1164768508 learned_addr learned my addr 172.21.15.83:0/1164768508 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731c04700 1 --2- 172.21.15.83:0/1164768508 >> v2:172.21.15.90:3300/0 conn(0x7f272c0b1900 0x7f272c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731403700 1 -- 172.21.15.83:0/1164768508 >> v2:172.21.15.90:3300/0 conn(0x7f272c0b1900 msgr2=0x7f272c0b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731403700 1 --2- 172.21.15.83:0/1164768508 >> v2:172.21.15.90:3300/0 conn(0x7f272c0b1900 0x7f272c0b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731403700 1 -- 172.21.15.83:0/1164768508 >> v1:172.21.15.90:6789/0 conn(0x7f272c15e000 legacy=0x7f272c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731403700 1 -- 172.21.15.83:0/1164768508 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c06c140 con 0x7f272c0b5600 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2730c02700 1 -- 172.21.15.83:0/1164768508 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064047128 0 0) 0x7f272c06bc10 con 0x7f272c15e000 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.619+0000 7f2731c04700 1 --2- 172.21.15.83:0/1164768508 >> v2:172.21.15.90:3300/0 conn(0x7f272c0b1900 0x7f272c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2731403700 1 --2- 172.21.15.83:0/1164768508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c0b5600 0x7f272c0cb330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f272c071980 tx=0x7f2728009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930d2478bb78d85c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2730c02700 1 -- 172.21.15.83:0/1164768508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728011830 con 0x7f272c0b5600 2023-11-28T11:09:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2730c02700 1 -- 172.21.15.83:0/1164768508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2728011990 con 0x7f272c0b5600 2023-11-28T11:09:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2730c02700 1 -- 172.21.15.83:0/1164768508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728011d20 con 0x7f272c0b5600 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 -- 172.21.15.83:0/1164768508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c0b5600 msgr2=0x7f272c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 --2- 172.21.15.83:0/1164768508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c0b5600 0x7f272c0cb330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f272c071980 tx=0x7f2728009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 -- 172.21.15.83:0/1164768508 shutdown_connections 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 --2- 172.21.15.83:0/1164768508 >> v2:172.21.15.90:3300/0 conn(0x7f272c0b1900 0x7f272c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 --2- 172.21.15.83:0/1164768508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c0b5600 0x7f272c0cb330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 -- 172.21.15.83:0/1164768508 >> 172.21.15.83:0/1164768508 conn(0x7f272c0cc690 msgr2=0x7f272c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.620+0000 7f2733e68700 1 -- 172.21.15.83:0/1164768508 shutdown_connections 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.621+0000 7f2733e68700 1 -- 172.21.15.83:0/1164768508 wait complete. 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.622+0000 7f2733e68700 1 Processor -- start 2023-11-28T11:09:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.622+0000 7f2733e68700 1 -- start start 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2733e68700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f272c0b1900 0x7f272c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2733e68700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c15e000 0x7f272c1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2733e68700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 0x7f272c149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2733e68700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c15bf10 con 0x7f272c15cc90 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2733e68700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c14a6b0 con 0x7f272c15e000 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2733e68700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0b4520 con 0x7f272c0b1900 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 0x7f272c149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 0x7f272c149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49400/0 (socket says 172.21.15.83:49400) 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 -- 172.21.15.83:0/3131696007 learned_addr learned my addr 172.21.15.83:0/3131696007 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2731403700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c15e000 0x7f272c1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2731c04700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f272c0b1900 0x7f272c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 -- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f272c0b1900 msgr2=0x7f272c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f272c0b1900 0x7f272c1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 -- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c15e000 msgr2=0x7f272c1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c15e000 0x7f272c1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2732405700 1 -- 172.21.15.83:0/3131696007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280094a0 con 0x7f272c15cc90 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.623+0000 7f2731403700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c15e000 0x7f272c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.624+0000 7f2732405700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 0x7f272c149720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f272400cc10 tx=0x7f27240021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.624+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272400ab80 con 0x7f272c15cc90 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.624+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272400ad00 con 0x7f272c15cc90 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.624+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c06c700 con 0x7f272c15cc90 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.624+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724007310 con 0x7f272c15cc90 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.624+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c06c420 con 0x7f272c15cc90 2023-11-28T11:09:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.625+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c0b1cd0 con 0x7f272c15cc90 2023-11-28T11:09:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.625+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f272401a030 con 0x7f272c15cc90 2023-11-28T11:09:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.625+0000 7f27227fc700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27100682f0 0x7f271006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.626+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2724085cb0 con 0x7f272c15cc90 2023-11-28T11:09:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.626+0000 7f2731c04700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27100682f0 0x7f271006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.626+0000 7f2731c04700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27100682f0 0x7f271006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f272c0cf930 tx=0x7f271c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.630+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2724052220 con 0x7f272c15cc90 2023-11-28T11:09:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.810+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f272c0742b0 con 0x7f272c15cc90 2023-11-28T11:09:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.810+0000 7f27227fc700 1 -- 172.21.15.83:0/3131696007 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27240180a0 con 0x7f272c15cc90 2023-11-28T11:09:10.812 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27100682f0 msgr2=0x7f271006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27100682f0 0x7f271006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f272c0cf930 tx=0x7f271c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 msgr2=0x7f272c149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 0x7f272c149720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f272400cc10 tx=0x7f27240021c0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 shutdown_connections 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27100682f0 0x7f271006a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272c15cc90 0x7f272c149720 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f272c0b1900 0x7f272c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 --2- 172.21.15.83:0/3131696007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f272c15e000 0x7f272c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.812+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 >> 172.21.15.83:0/3131696007 conn(0x7f272c0cc690 msgr2=0x7f272c0cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.813+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 shutdown_connections 2023-11-28T11:09:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.813+0000 7f2733e68700 1 -- 172.21.15.83:0/3131696007 wait complete. 2023-11-28T11:09:10.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.948+0000 7f2f250f0700 1 Processor -- start 2023-11-28T11:09:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.948+0000 7f2f250f0700 1 -- start start 2023-11-28T11:09:10.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f250f0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2f200b1900 0x7f2f200b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f250f0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f200b5600 0x7f2f200cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f250f0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2006bc10 con 0x7f2f2015e000 2023-11-28T11:09:10.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f250f0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2006cc20 con 0x7f2f200b5600 2023-11-28T11:09:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f250f0700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2006be60 con 0x7f2f200b1900 2023-11-28T11:09:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f200b5600 0x7f2f200cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1f59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2f2015e000 0x7f2f200cb870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47268/0 (socket says 172.21.15.83:47268) 2023-11-28T11:09:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1f59e700 1 -- 172.21.15.83:0/2170935368 learned_addr learned my addr 172.21.15.83:0/2170935368 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1ed9d700 1 --2- 172.21.15.83:0/2170935368 >> v2:172.21.15.90:3300/0 conn(0x7f2f200b1900 0x7f2f200b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1e59c700 1 -- 172.21.15.83:0/2170935368 >> v2:172.21.15.90:3300/0 conn(0x7f2f200b1900 msgr2=0x7f2f200b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1e59c700 1 --2- 172.21.15.83:0/2170935368 >> v2:172.21.15.90:3300/0 conn(0x7f2f200b1900 0x7f2f200b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1e59c700 1 -- 172.21.15.83:0/2170935368 >> v1:172.21.15.83:6789/0 conn(0x7f2f2015e000 legacy=0x7f2f200cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1e59c700 1 -- 172.21.15.83:0/2170935368 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2006c140 con 0x7f2f200b5600 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1dd9b700 1 -- 172.21.15.83:0/2170935368 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847624986 0 0) 0x7f2f2006bc10 con 0x7f2f2015e000 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.949+0000 7f2f1e59c700 1 --2- 172.21.15.83:0/2170935368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200b5600 0x7f2f200cb330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f20071980 tx=0x7f2f10009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb3b5255fa75f2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f1dd9b700 1 -- 172.21.15.83:0/2170935368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10011830 con 0x7f2f200b5600 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f1dd9b700 1 -- 172.21.15.83:0/2170935368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f10011990 con 0x7f2f200b5600 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f1dd9b700 1 -- 172.21.15.83:0/2170935368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10011d20 con 0x7f2f200b5600 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 -- 172.21.15.83:0/2170935368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200b5600 msgr2=0x7f2f200cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2170935368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200b5600 0x7f2f200cb330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f20071980 tx=0x7f2f10009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 -- 172.21.15.83:0/2170935368 shutdown_connections 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2170935368 >> v2:172.21.15.90:3300/0 conn(0x7f2f200b1900 0x7f2f200b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2170935368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200b5600 0x7f2f200cb330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 -- 172.21.15.83:0/2170935368 >> 172.21.15.83:0/2170935368 conn(0x7f2f200cc690 msgr2=0x7f2f200cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 -- 172.21.15.83:0/2170935368 shutdown_connections 2023-11-28T11:09:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.950+0000 7f2f250f0700 1 -- 172.21.15.83:0/2170935368 wait complete. 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.952+0000 7f2f250f0700 1 Processor -- start 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.952+0000 7f2f250f0700 1 -- start start 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f250f0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f200b1900 0x7f2f20148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f250f0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2015e000 0x7f2f200ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f250f0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 0x7f2f200ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f250f0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200c3720 con 0x7f2f200c4780 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f250f0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200c39c0 con 0x7f2f2015e000 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f250f0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200c3ca0 con 0x7f2f200b1900 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 0x7f2f200ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 0x7f2f200ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49414/0 (socket says 172.21.15.83:49414) 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 -- 172.21.15.83:0/2008792850 learned_addr learned my addr 172.21.15.83:0/2008792850 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1ed9d700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f200b1900 0x7f2f20148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1e59c700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2015e000 0x7f2f200ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 -- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f200b1900 msgr2=0x7f2f20148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f200b1900 0x7f2f20148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 -- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2015e000 msgr2=0x7f2f200ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.953+0000 7f2f1f59e700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2015e000 0x7f2f200ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f1f59e700 1 -- 172.21.15.83:0/2008792850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100094a0 con 0x7f2f200c4780 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f1ed9d700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f200b1900 0x7f2f20148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f1f59e700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 0x7f2f200ced50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800cc10 tx=0x7f2f1800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800ab50 con 0x7f2f200c4780 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1800acd0 con 0x7f2f200c4780 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18007310 con 0x7f2f200c4780 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2006c700 con 0x7f2f200c4780 2023-11-28T11:09:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.954+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2006c420 con 0x7f2f200c4780 2023-11-28T11:09:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.955+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f200b1cd0 con 0x7f2f200c4780 2023-11-28T11:09:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.955+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f18010070 con 0x7f2f200c4780 2023-11-28T11:09:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.956+0000 7f2f077fe700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f0c0682f0 0x7f2f0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.956+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2f18085800 con 0x7f2f200c4780 2023-11-28T11:09:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.956+0000 7f2f1ed9d700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f0c0682f0 0x7f2f0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.956+0000 7f2f1ed9d700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f0c0682f0 0x7f2f0c06a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f20153600 tx=0x7f2f14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:10.960+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f18051d70 con 0x7f2f200c4780 2023-11-28T11:09:11.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.143+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f200742b0 con 0x7f2f200c4780 2023-11-28T11:09:11.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.143+0000 7f2f077fe700 1 -- 172.21.15.83:0/2008792850 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f18055390 con 0x7f2f200c4780 2023-11-28T11:09:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:11.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f0c0682f0 msgr2=0x7f2f0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f0c0682f0 0x7f2f0c06a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f20153600 tx=0x7f2f14008040 comp rx=0 tx=0).stop 2023-11-28T11:09:11.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 msgr2=0x7f2f200ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 0x7f2f200ced50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800cc10 tx=0x7f2f1800cf20 comp rx=0 tx=0).stop 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 shutdown_connections 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f0c0682f0 0x7f2f0c06a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f200c4780 0x7f2f200ced50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f200b1900 0x7f2f20148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 --2- 172.21.15.83:0/2008792850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2015e000 0x7f2f200ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 >> 172.21.15.83:0/2008792850 conn(0x7f2f200cc690 msgr2=0x7f2f2006ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 shutdown_connections 2023-11-28T11:09:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.146+0000 7f2f250f0700 1 -- 172.21.15.83:0/2008792850 wait complete. 2023-11-28T11:09:11.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:11.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.280+0000 7f6abe697700 1 Processor -- start 2023-11-28T11:09:11.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.280+0000 7f6abe697700 1 -- start start 2023-11-28T11:09:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6abe697700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ab80aee20 0x7f6ab80b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6abe697700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6ab806c370 0x7f6ab806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6abe697700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab80b5b00 con 0x7f6ab80af1f0 2023-11-28T11:09:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6abe697700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab806c800 con 0x7f6ab80aee20 2023-11-28T11:09:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6abe697700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab806cae0 con 0x7f6ab806c370 2023-11-28T11:09:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6ab7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ab80aee20 0x7f6ab80b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6ab7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ab80aee20 0x7f6ab80b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49418/0 (socket says 172.21.15.83:49418) 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2580874600 learned_addr learned my addr 172.21.15.83:0/2580874600 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6ab77fe700 1 --2- 172.21.15.83:0/2580874600 >> v2:172.21.15.90:3300/0 conn(0x7f6ab806c370 0x7f6ab806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.281+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2580874600 >> v2:172.21.15.90:3300/0 conn(0x7f6ab806c370 msgr2=0x7f6ab806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab7fff700 1 --2- 172.21.15.83:0/2580874600 >> v2:172.21.15.90:3300/0 conn(0x7f6ab806c370 0x7f6ab806ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2580874600 >> v1:172.21.15.90:6789/0 conn(0x7f6ab80af1f0 legacy=0x7f6ab80cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2580874600 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab806cdc0 con 0x7f6ab80aee20 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab7fff700 1 --2- 172.21.15.83:0/2580874600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab80aee20 0x7f6ab80b55c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80b3bb0 tx=0x7f6aa0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c3e1ca5d2d5f2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab6ffd700 1 -- 172.21.15.83:0/2580874600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0005e30 con 0x7f6ab80aee20 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab6ffd700 1 -- 172.21.15.83:0/2580874600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0006030 con 0x7f6ab80aee20 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6ab6ffd700 1 -- 172.21.15.83:0/2580874600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00063e0 con 0x7f6ab80aee20 2023-11-28T11:09:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 -- 172.21.15.83:0/2580874600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab80aee20 msgr2=0x7f6ab80b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 --2- 172.21.15.83:0/2580874600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab80aee20 0x7f6ab80b55c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80b3bb0 tx=0x7f6aa0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 -- 172.21.15.83:0/2580874600 shutdown_connections 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 --2- 172.21.15.83:0/2580874600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab80aee20 0x7f6ab80b55c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 --2- 172.21.15.83:0/2580874600 >> v2:172.21.15.90:3300/0 conn(0x7f6ab806c370 0x7f6ab806ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 -- 172.21.15.83:0/2580874600 >> 172.21.15.83:0/2580874600 conn(0x7f6ab80b2b60 msgr2=0x7f6ab80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 -- 172.21.15.83:0/2580874600 shutdown_connections 2023-11-28T11:09:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.282+0000 7f6abe697700 1 -- 172.21.15.83:0/2580874600 wait complete. 2023-11-28T11:09:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 Processor -- start 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 -- start start 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 0x7f6ab80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ab80aee20 0x7f6ab80d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ab80af1f0 0x7f6ab814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80ae650 con 0x7f6ab806c370 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81f5f10 con 0x7f6ab80aee20 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.284+0000 7f6abe697700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81f61f0 con 0x7f6ab80af1f0 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 0x7f6ab80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 0x7f6ab80adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49424/0 (socket says 172.21.15.83:49424) 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2934147634 learned_addr learned my addr 172.21.15.83:0/2934147634 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ab80af1f0 msgr2=0x7f6ab814ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ab80af1f0 0x7f6ab814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ab80aee20 msgr2=0x7f6ab80d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab77fe700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ab80aee20 0x7f6ab80d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ab80aee20 0x7f6ab80d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 -- 172.21.15.83:0/2934147634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0009500 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab7fff700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 0x7f6ab80adcd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006000 tx=0x7f6aa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0004250 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0006980 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab806fc70 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00045f0 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.285+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab806d0a0 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.286+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab806d380 con 0x7f6ab806c370 2023-11-28T11:09:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.287+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6aa000f050 con 0x7f6ab806c370 2023-11-28T11:09:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.287+0000 7f6ab4ff9700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a980682f0 0x7f6a9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.287+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6aa0087960 con 0x7f6ab806c370 2023-11-28T11:09:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.287+0000 7f6ab77fe700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a980682f0 0x7f6a9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.287+0000 7f6ab77fe700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a980682f0 0x7f6a9806a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8156950 tx=0x7f6aa8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.291+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6aa0053ed0 con 0x7f6ab806c370 2023-11-28T11:09:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.471+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ab80b5b00 con 0x7f6ab806c370 2023-11-28T11:09:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.472+0000 7f6ab4ff9700 1 -- 172.21.15.83:0/2934147634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ab80b5b00 con 0x7f6ab806c370 2023-11-28T11:09:11.474 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:11.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.473+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a980682f0 msgr2=0x7f6a9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a980682f0 0x7f6a9806a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8156950 tx=0x7f6aa8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 msgr2=0x7f6ab80adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 0x7f6ab80adcd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006000 tx=0x7f6aa0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 shutdown_connections 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ab80af1f0 0x7f6ab814ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a980682f0 0x7f6a9806a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ab806c370 0x7f6ab80adcd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 --2- 172.21.15.83:0/2934147634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ab80aee20 0x7f6ab80d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 >> 172.21.15.83:0/2934147634 conn(0x7f6ab80b2b60 msgr2=0x7f6ab806ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 shutdown_connections 2023-11-28T11:09:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.474+0000 7f6abe697700 1 -- 172.21.15.83:0/2934147634 wait complete. 2023-11-28T11:09:11.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.615+0000 7f980a951700 1 Processor -- start 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.615+0000 7f980a951700 1 -- start start 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f980a951700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f980406ea70 0x7f980406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f980a951700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f980a951700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98040cc4b0 con 0x7f980406e6a0 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f980a951700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980406f420 con 0x7f980406ea70 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f980a951700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9804160ec0 con 0x7f9804160500 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f980406ea70 0x7f980406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f980406ea70 0x7f980406eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49434/0 (socket says 172.21.15.83:49434) 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 -- 172.21.15.83:0/2192835523 learned_addr learned my addr 172.21.15.83:0/2192835523 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f9808eee700 1 --2- 172.21.15.83:0/2192835523 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 -- 172.21.15.83:0/2192835523 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 msgr2=0x7f9804160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 --2- 172.21.15.83:0/2192835523 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 -- 172.21.15.83:0/2192835523 >> v1:172.21.15.90:6789/0 conn(0x7f980406e6a0 legacy=0x7f98040cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 -- 172.21.15.83:0/2192835523 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040cc790 con 0x7f980406ea70 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f98037fe700 1 --2- 172.21.15.83:0/2192835523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f980406eee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f97f40098f0 tx=0x7f97f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2849f95d7ffd9a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.616+0000 7f9802ffd700 1 -- 172.21.15.83:0/2192835523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400b8a0 con 0x7f980406ea70 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f9802ffd700 1 -- 172.21.15.83:0/2192835523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f400ba00 con 0x7f980406ea70 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f9802ffd700 1 -- 172.21.15.83:0/2192835523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400bd90 con 0x7f980406ea70 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 -- 172.21.15.83:0/2192835523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 msgr2=0x7f980406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 --2- 172.21.15.83:0/2192835523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f980406eee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f97f40098f0 tx=0x7f97f4009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 -- 172.21.15.83:0/2192835523 shutdown_connections 2023-11-28T11:09:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 --2- 172.21.15.83:0/2192835523 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 --2- 172.21.15.83:0/2192835523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f980406eee0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 -- 172.21.15.83:0/2192835523 >> 172.21.15.83:0/2192835523 conn(0x7f98040b2b80 msgr2=0x7f98040b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 -- 172.21.15.83:0/2192835523 shutdown_connections 2023-11-28T11:09:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.617+0000 7f980a951700 1 -- 172.21.15.83:0/2192835523 wait complete. 2023-11-28T11:09:11.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.618+0000 7f980a951700 1 Processor -- start 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.618+0000 7f980a951700 1 -- start start 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f980a951700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f980406e6a0 0x7f98041524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f980a951700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f9804157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f980a951700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f980a951700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980415d8b0 con 0x7f980406ea70 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f980a951700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98041529e0 con 0x7f980406e6a0 2023-11-28T11:09:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f980a951700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98041f6540 con 0x7f9804160500 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f98037fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f9804157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f98037fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f9804157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49440/0 (socket says 172.21.15.83:49440) 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f98037fe700 1 -- 172.21.15.83:0/1262488329 learned_addr learned my addr 172.21.15.83:0/1262488329 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f9803fff700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f980406e6a0 0x7f98041524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.619+0000 7f9808eee700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f98037fe700 1 -- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 msgr2=0x7f9804157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f98037fe700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f98037fe700 1 -- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f980406e6a0 msgr2=0x7f98041524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f98037fe700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f980406e6a0 0x7f98041524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f98037fe700 1 -- 172.21.15.83:0/1262488329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4009550 con 0x7f980406ea70 2023-11-28T11:09:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f9808eee700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f98037fe700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f9804157760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97f4000c00 tx=0x7f97f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4004480 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f401f070 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40038c0 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98040d0540 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.620+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98040d0260 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.621+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98040d0820 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.621+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97f402a020 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.622+0000 7f9800ff9700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.622+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97f40874d0 con 0x7f980406ea70 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.622+0000 7f9803fff700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.622+0000 7f9803fff700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f97ec000fb0 tx=0x7f97ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.626+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97f4053a40 con 0x7f980406ea70 2023-11-28T11:09:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.809+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f98040742b0 con 0x7f980406ea70 2023-11-28T11:09:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.810+0000 7f9800ff9700 1 -- 172.21.15.83:0/1262488329 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f97f4057060 con 0x7f980406ea70 2023-11-28T11:09:11.811 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 msgr2=0x7f97e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f97ec000fb0 tx=0x7f97ec009040 comp rx=0 tx=0).stop 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 msgr2=0x7f9804157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f9804157760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97f4000c00 tx=0x7f97f4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 shutdown_connections 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9804160500 0x7f9804157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f980406e6a0 0x7f98041524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97e40682f0 0x7f97e406a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 --2- 172.21.15.83:0/1262488329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f980406ea70 0x7f9804157760 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 >> 172.21.15.83:0/1262488329 conn(0x7f98040b2b80 msgr2=0x7f98040ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 shutdown_connections 2023-11-28T11:09:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.812+0000 7f980a951700 1 -- 172.21.15.83:0/1262488329 wait complete. 2023-11-28T11:09:11.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.944+0000 7fc2f30fc700 1 Processor -- start 2023-11-28T11:09:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.944+0000 7fc2f30fc700 1 -- start start 2023-11-28T11:09:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2f30fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2ec0b3070 0x7fc2ec0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2f30fc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec15f1d0 con 0x7fc2ec15a8e0 2023-11-28T11:09:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2f30fc700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec15eef0 con 0x7fc2ec15a510 2023-11-28T11:09:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2f30fc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cd780 con 0x7fc2ec0b3070 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2ebfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc2ec15a8e0 0x7fc2ec0b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47308/0 (socket says 172.21.15.83:47308) 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2ebfff700 1 -- 172.21.15.83:0/1629224669 learned_addr learned my addr 172.21.15.83:0/1629224669 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2f1699700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2ec0b3070 0x7fc2ec0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024440044 0 0) 0x7fc2ec15f1d0 con 0x7fc2ec15a8e0 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc0035b0 con 0x7fc2ec15a8e0 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895820510 0 0) 0x7fc2ec15eef0 con 0x7fc2ec15a510 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ec15f1d0 con 0x7fc2ec15a510 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2073294530 0 0) 0x7fc2dc0035b0 con 0x7fc2ec15a8e0 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2ec0b3070 msgr2=0x7fc2ec0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2ec0b3070 0x7fc2ec0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 >> v1:172.21.15.90:6789/0 conn(0x7fc2ec15a510 legacy=0x7fc2ec0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0cedd0 con 0x7fc2ec15a8e0 2023-11-28T11:09:11.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fc2d8001670 con 0x7fc2ec15a8e0 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 >> v1:172.21.15.83:6789/0 conn(0x7fc2ec15a8e0 legacy=0x7fc2ec0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2dc003bc0 0x7fc2dc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2dc004850 0x7fc2dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc005200 0x7fc2dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00bfa0 con 0x7fc2dc004850 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00c110 con 0x7fc2dc003bc0 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00c2d0 con 0x7fc2dc005200 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2f1699700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2ec0b3070 0x7fc2ec0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.945+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2dc004850 0x7fc2dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2f0e98700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2dc003bc0 0x7fc2dc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2f1699700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc005200 0x7fc2dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2ebfff700 1 -- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc005200 msgr2=0x7fc2dc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc005200 0x7fc2dc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2ebfff700 1 -- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2dc003bc0 msgr2=0x7fc2dc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2dc003bc0 0x7fc2dc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2ebfff700 1 -- 172.21.15.83:0/1629224669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0035b0 con 0x7fc2dc004850 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2f1699700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc005200 0x7fc2dc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2dc004850 0x7fc2dc004cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c210 tx=0x7fc2dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e596b1e17a998d52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800cc90 con 0x7fc2dc004850 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800ce10 con 0x7fc2dc004850 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2eb7fe700 1 -- 172.21.15.83:0/1629224669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8005ba0 con 0x7fc2dc004850 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2f30fc700 1 -- 172.21.15.83:0/1629224669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2dc004850 msgr2=0x7fc2dc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/1629224669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2dc004850 0x7fc2dc004cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c210 tx=0x7fc2dc005880 comp rx=0 tx=0).stop 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.946+0000 7fc2f1699700 1 -- 172.21.15.83:0/1629224669 reap_dead start 2023-11-28T11:09:11.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.947+0000 7fc2f30fc700 1 -- 172.21.15.83:0/1629224669 shutdown_connections 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.947+0000 7fc2f30fc700 1 -- 172.21.15.83:0/1629224669 >> 172.21.15.83:0/1629224669 conn(0x7fc2ec0cd960 msgr2=0x7fc2ec0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.947+0000 7fc2f30fc700 1 -- 172.21.15.83:0/1629224669 shutdown_connections 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.947+0000 7fc2f30fc700 1 -- 172.21.15.83:0/1629224669 wait complete. 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.948+0000 7fc2f30fc700 1 Processor -- start 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.948+0000 7fc2f30fc700 1 -- start start 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc004850 0x7fc2ec0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 0x7fc2ec0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2ec0b1e50 0x7fc2ec0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cb4f0 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cb210 con 0x7fc2ec0b1e50 2023-11-28T11:09:11.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0c4b00 con 0x7fc2dc004850 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 0x7fc2ec0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 0x7fc2ec0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49460/0 (socket says 172.21.15.83:49460) 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 -- 172.21.15.83:0/3085140977 learned_addr learned my addr 172.21.15.83:0/3085140977 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 -- 172.21.15.83:0/3085140977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc004850 msgr2=0x7fc2ec0c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc004850 0x7fc2ec0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 -- 172.21.15.83:0/3085140977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2ec0b1e50 msgr2=0x7fc2ec0c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2ec0b1e50 0x7fc2ec0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 -- 172.21.15.83:0/3085140977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0cf390 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2ebfff700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 0x7fc2ec0c7fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c0e0 tx=0x7fc2d8001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80072e0 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d8007440 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec0cf0b0 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80077d0 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.949+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec0cffc0 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.950+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec074320 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.951+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2d8024030 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.951+0000 7fc2e97fa700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc2d00682f0 0x7fc2d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.951+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc2d8086fb0 con 0x7fc2ec0cacc0 2023-11-28T11:09:11.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.951+0000 7fc2f0e98700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc2d00682f0 0x7fc2d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.952+0000 7fc2f0e98700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc2d00682f0 0x7fc2d006a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0007fd0 tx=0x7fc2e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:11.958+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2d80535e0 con 0x7fc2ec0cacc0 2023-11-28T11:09:12.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.144+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2ec0b6a00 con 0x7fc2ec0cacc0 2023-11-28T11:09:12.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.145+0000 7fc2e97fa700 1 -- 172.21.15.83:0/3085140977 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fc2d8056c00 con 0x7fc2ec0cacc0 2023-11-28T11:09:12.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:12.147 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.147+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc2d00682f0 msgr2=0x7fc2d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.147+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc2d00682f0 0x7fc2d006a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0007fd0 tx=0x7fc2e0009040 comp rx=0 tx=0).stop 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.147+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 msgr2=0x7fc2ec0c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.147+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 0x7fc2ec0c7fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c0e0 tx=0x7fc2d8001980 comp rx=0 tx=0).stop 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 shutdown_connections 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2ec0b1e50 0x7fc2ec0c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2dc004850 0x7fc2ec0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc2d00682f0 0x7fc2d006a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 --2- 172.21.15.83:0/3085140977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ec0cacc0 0x7fc2ec0c7fb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 >> 172.21.15.83:0/3085140977 conn(0x7fc2ec0cd960 msgr2=0x7fc2ec0b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 shutdown_connections 2023-11-28T11:09:12.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.148+0000 7fc2f30fc700 1 -- 172.21.15.83:0/3085140977 wait complete. 2023-11-28T11:09:12.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:12.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:12.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.283+0000 7f945286c700 1 Processor -- start 2023-11-28T11:09:12.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.283+0000 7f945286c700 1 -- start start 2023-11-28T11:09:12.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f945286c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f944c156d70 0x7f944c0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f945286c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c0d0bf0 0x7f944c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f945286c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f944c15d1a0 0x7f944c160250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f945286c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c15dcc0 con 0x7f944c156d70 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f945286c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c157140 con 0x7f944c15d1a0 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f945286c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c15d9e0 con 0x7f944c0d0bf0 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f944c156d70 0x7f944c0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f944c156d70 0x7f944c0d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49466/0 (socket says 172.21.15.83:49466) 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 -- 172.21.15.83:0/3507814355 learned_addr learned my addr 172.21.15.83:0/3507814355 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f9450e09700 1 --2- 172.21.15.83:0/3507814355 >> v2:172.21.15.90:3300/0 conn(0x7f944c15d1a0 0x7f944c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944b7fe700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c0d0bf0 0x7f944c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 -- 172.21.15.83:0/3507814355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c0d0bf0 msgr2=0x7f944c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c0d0bf0 0x7f944c15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 -- 172.21.15.83:0/3507814355 >> v2:172.21.15.90:3300/0 conn(0x7f944c15d1a0 msgr2=0x7f944c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 --2- 172.21.15.83:0/3507814355 >> v2:172.21.15.90:3300/0 conn(0x7f944c15d1a0 0x7f944c160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f944bfff700 1 -- 172.21.15.83:0/3507814355 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c06bdb0 con 0x7f944c156d70 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.284+0000 7f9450e09700 1 --2- 172.21.15.83:0/3507814355 >> v2:172.21.15.90:3300/0 conn(0x7f944c15d1a0 0x7f944c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f944b7fe700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c0d0bf0 0x7f944c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f944bfff700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c156d70 0x7f944c0d06b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f944c0cb2c0 tx=0x7f943c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fced043c68ecc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f944affd700 1 -- 172.21.15.83:0/3507814355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00b870 con 0x7f944c156d70 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f944affd700 1 -- 172.21.15.83:0/3507814355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c00b9d0 con 0x7f944c156d70 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f944affd700 1 -- 172.21.15.83:0/3507814355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00bd60 con 0x7f944c156d70 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 -- 172.21.15.83:0/3507814355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c156d70 msgr2=0x7f944c0d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c156d70 0x7f944c0d06b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f944c0cb2c0 tx=0x7f943c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 -- 172.21.15.83:0/3507814355 shutdown_connections 2023-11-28T11:09:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 --2- 172.21.15.83:0/3507814355 >> v2:172.21.15.90:3300/0 conn(0x7f944c15d1a0 0x7f944c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c156d70 0x7f944c0d06b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 --2- 172.21.15.83:0/3507814355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c0d0bf0 0x7f944c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 -- 172.21.15.83:0/3507814355 >> 172.21.15.83:0/3507814355 conn(0x7f944c06ea50 msgr2=0x7f944c070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 -- 172.21.15.83:0/3507814355 shutdown_connections 2023-11-28T11:09:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.285+0000 7f945286c700 1 -- 172.21.15.83:0/3507814355 wait complete. 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.287+0000 7f945286c700 1 Processor -- start 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.287+0000 7f945286c700 1 -- start start 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f945286c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944c156d70 0x7f944c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f945286c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c15d1a0 0x7f944c1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f945286c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 0x7f944c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f945286c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c15b800 con 0x7f944c15ae40 2023-11-28T11:09:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f945286c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0cad50 con 0x7f944c156d70 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f945286c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c1575d0 con 0x7f944c15d1a0 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 0x7f944c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 0x7f944c15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49476/0 (socket says 172.21.15.83:49476) 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 -- 172.21.15.83:0/1810057708 learned_addr learned my addr 172.21.15.83:0/1810057708 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f944bfff700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944c156d70 0x7f944c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f944b7fe700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c15d1a0 0x7f944c1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 -- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c15d1a0 msgr2=0x7f944c1f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c15d1a0 0x7f944c1f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 -- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944c156d70 msgr2=0x7f944c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944c156d70 0x7f944c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f9450e09700 1 -- 172.21.15.83:0/1810057708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c009550 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.288+0000 7f944b7fe700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c15d1a0 0x7f944c1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.289+0000 7f9450e09700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 0x7f944c15b2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f944400cef0 tx=0x7f944400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.289+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400adb0 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.289+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444015440 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.289+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440157f0 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.289+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c06c370 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.289+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c06c090 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.290+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c0cf1e0 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.290+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f944401c030 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.291+0000 7f9448ff9700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94380682f0 0x7f943806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.291+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9444086300 con 0x7f944c15ae40 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.291+0000 7f944bfff700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94380682f0 0x7f943806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.291+0000 7f944bfff700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94380682f0 0x7f943806a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.295+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9444052870 con 0x7f944c15ae40 2023-11-28T11:09:12.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.479+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f944c0742b0 con 0x7f944c15ae40 2023-11-28T11:09:12.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.480+0000 7f9448ff9700 1 -- 172.21.15.83:0/1810057708 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f944401a020 con 0x7f944c15ae40 2023-11-28T11:09:12.481 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94380682f0 msgr2=0x7f943806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94380682f0 0x7f943806a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 msgr2=0x7f944c15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 0x7f944c15b2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f944400cef0 tx=0x7f944400cf20 comp rx=0 tx=0).stop 2023-11-28T11:09:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 shutdown_connections 2023-11-28T11:09:12.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94380682f0 0x7f943806a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f944c15ae40 0x7f944c15b2c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944c156d70 0x7f944c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 --2- 172.21.15.83:0/1810057708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944c15d1a0 0x7f944c1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 >> 172.21.15.83:0/1810057708 conn(0x7f944c0d0bf0 msgr2=0x7f944c160690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 shutdown_connections 2023-11-28T11:09:12.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.482+0000 7f945286c700 1 -- 172.21.15.83:0/1810057708 wait complete. 2023-11-28T11:09:12.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.617+0000 7f1516e4d700 1 Processor -- start 2023-11-28T11:09:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.617+0000 7f1516e4d700 1 -- start start 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1516e4d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1510159450 0x7f151015d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1516e4d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1510070ad0 0x7f1510070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1516e4d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f1510159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1516e4d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100714b0 con 0x7f1510159450 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1516e4d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151015deb0 con 0x7f1510070ad0 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1516e4d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151015cf20 con 0x7f1510159820 2023-11-28T11:09:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1510159450 0x7f151015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1510159450 0x7f151015d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49480/0 (socket says 172.21.15.83:49480) 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 -- 172.21.15.83:0/2046518379 learned_addr learned my addr 172.21.15.83:0/2046518379 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f15153ea700 1 --2- 172.21.15.83:0/2046518379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f1510159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f150ffff700 1 --2- 172.21.15.83:0/2046518379 >> v2:172.21.15.90:3300/0 conn(0x7f1510070ad0 0x7f1510070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 -- 172.21.15.83:0/2046518379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 msgr2=0x7f1510159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 --2- 172.21.15.83:0/2046518379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f1510159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 -- 172.21.15.83:0/2046518379 >> v2:172.21.15.90:3300/0 conn(0x7f1510070ad0 msgr2=0x7f1510070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 --2- 172.21.15.83:0/2046518379 >> v2:172.21.15.90:3300/0 conn(0x7f1510070ad0 0x7f1510070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f1514be9700 1 -- 172.21.15.83:0/2046518379 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151015a1c0 con 0x7f1510159450 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.618+0000 7f150ffff700 1 --2- 172.21.15.83:0/2046518379 >> v2:172.21.15.90:3300/0 conn(0x7f1510070ad0 0x7f1510070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1514be9700 1 --2- 172.21.15.83:0/2046518379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151015d970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f15040098f0 tx=0x7f1504009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd81c322ae81b8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f150f7fe700 1 -- 172.21.15.83:0/2046518379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400b860 con 0x7f1510159450 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f150f7fe700 1 -- 172.21.15.83:0/2046518379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150400b9c0 con 0x7f1510159450 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f150f7fe700 1 -- 172.21.15.83:0/2046518379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400bd50 con 0x7f1510159450 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 -- 172.21.15.83:0/2046518379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 msgr2=0x7f151015d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 --2- 172.21.15.83:0/2046518379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151015d970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f15040098f0 tx=0x7f1504009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 -- 172.21.15.83:0/2046518379 shutdown_connections 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 --2- 172.21.15.83:0/2046518379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f1510159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 --2- 172.21.15.83:0/2046518379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151015d970 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 --2- 172.21.15.83:0/2046518379 >> v2:172.21.15.90:3300/0 conn(0x7f1510070ad0 0x7f1510070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 -- 172.21.15.83:0/2046518379 >> 172.21.15.83:0/2046518379 conn(0x7f15100ceb80 msgr2=0x7f15100cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 -- 172.21.15.83:0/2046518379 shutdown_connections 2023-11-28T11:09:12.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.619+0000 7f1516e4d700 1 -- 172.21.15.83:0/2046518379 wait complete. 2023-11-28T11:09:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.620+0000 7f1516e4d700 1 Processor -- start 2023-11-28T11:09:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 -- start start 2023-11-28T11:09:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510070ad0 0x7f15101522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f151006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1510153e00 con 0x7f1510159450 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101540e0 con 0x7f1510070ad0 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f1516e4d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15101543c0 con 0x7f1510159820 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f150ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f150ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49488/0 (socket says 172.21.15.83:49488) 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.621+0000 7f150ffff700 1 -- 172.21.15.83:0/4027683227 learned_addr learned my addr 172.21.15.83:0/4027683227 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f1514be9700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510070ad0 0x7f15101522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f15153ea700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f151006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150ffff700 1 -- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 msgr2=0x7f151006a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150ffff700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f151006a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150ffff700 1 -- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510070ad0 msgr2=0x7f15101522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150ffff700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510070ad0 0x7f15101522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150ffff700 1 -- 172.21.15.83:0/4027683227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504009550 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150ffff700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151006a270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00d800 tx=0x7f14fc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc004030 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc004190 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151015a760 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc004520 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.622+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151015a480 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.623+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151015d170 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.624+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f14fc01a020 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.624+0000 7f150d7fa700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.624+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f14fc0863b0 con 0x7f1510159450 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.624+0000 7f1514be9700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.624+0000 7f1514be9700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1504009c40 tx=0x7f1504008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.629+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f14fc0528c0 con 0x7f1510159450 2023-11-28T11:09:12.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.811+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1510153f40 con 0x7f1510159450 2023-11-28T11:09:12.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.812+0000 7f150d7fa700 1 -- 172.21.15.83:0/4027683227 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1510153f40 con 0x7f1510159450 2023-11-28T11:09:12.813 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 msgr2=0x7f14f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1504009c40 tx=0x7f1504008030 comp rx=0 tx=0).stop 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 msgr2=0x7f151006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151006a270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00d800 tx=0x7f14fc00dbc0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 shutdown_connections 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1510159820 0x7f151006a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1510070ad0 0x7f15101522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14f40682f0 0x7f14f406a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 --2- 172.21.15.83:0/4027683227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1510159450 0x7f151006a270 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 >> 172.21.15.83:0/4027683227 conn(0x7f15100ceb80 msgr2=0x7f151006e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 shutdown_connections 2023-11-28T11:09:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.814+0000 7f1516e4d700 1 -- 172.21.15.83:0/4027683227 wait complete. 2023-11-28T11:09:12.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.949+0000 7fcc93d42700 1 Processor -- start 2023-11-28T11:09:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.949+0000 7fcc93d42700 1 -- start start 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc93d42700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc8c158900 0x7fcc8c159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc93d42700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcc8c158cd0 0x7fcc8c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc93d42700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8c0c9e70 con 0x7fcc8c0cb650 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc93d42700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8c06a980 con 0x7fcc8c158900 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc93d42700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8c15a000 con 0x7fcc8c158cd0 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc91ade700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc8c158900 0x7fcc8c159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc922df700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcc8c0cb650 0x7fcc8c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47358/0 (socket says 172.21.15.83:47358) 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc91ade700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc8c158900 0x7fcc8c159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49494/0 (socket says 172.21.15.83:49494) 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc922df700 1 -- 172.21.15.83:0/894684635 learned_addr learned my addr 172.21.15.83:0/894684635 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc91ade700 1 -- 172.21.15.83:0/894684635 >> v2:172.21.15.90:3300/0 conn(0x7fcc8c158cd0 msgr2=0x7fcc8c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc912dd700 1 --2- 172.21.15.83:0/894684635 >> v2:172.21.15.90:3300/0 conn(0x7fcc8c158cd0 0x7fcc8c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc91ade700 1 --2- 172.21.15.83:0/894684635 >> v2:172.21.15.90:3300/0 conn(0x7fcc8c158cd0 0x7fcc8c0cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc90adc700 1 -- 172.21.15.83:0/894684635 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071985042 0 0) 0x7fcc8c0c9e70 con 0x7fcc8c0cb650 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc91ade700 1 -- 172.21.15.83:0/894684635 >> v1:172.21.15.83:6789/0 conn(0x7fcc8c0cb650 legacy=0x7fcc8c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.950+0000 7fcc91ade700 1 -- 172.21.15.83:0/894684635 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c15c790 con 0x7fcc8c158900 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc91ade700 1 --2- 172.21.15.83:0/894684635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c159ac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0c99b0 tx=0x7fcc7c00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3a53083759b78a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc90adc700 1 -- 172.21.15.83:0/894684635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00dc70 con 0x7fcc8c158900 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc90adc700 1 -- 172.21.15.83:0/894684635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c00ddd0 con 0x7fcc8c158900 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc90adc700 1 -- 172.21.15.83:0/894684635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c010690 con 0x7fcc8c158900 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 -- 172.21.15.83:0/894684635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 msgr2=0x7fcc8c159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 --2- 172.21.15.83:0/894684635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c159ac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0c99b0 tx=0x7fcc7c00bdc0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 -- 172.21.15.83:0/894684635 shutdown_connections 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 --2- 172.21.15.83:0/894684635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c159ac0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 --2- 172.21.15.83:0/894684635 >> v2:172.21.15.90:3300/0 conn(0x7fcc8c158cd0 0x7fcc8c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 -- 172.21.15.83:0/894684635 >> 172.21.15.83:0/894684635 conn(0x7fcc8c0b1090 msgr2=0x7fcc8c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 -- 172.21.15.83:0/894684635 shutdown_connections 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.951+0000 7fcc93d42700 1 -- 172.21.15.83:0/894684635 wait complete. 2023-11-28T11:09:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.952+0000 7fcc93d42700 1 Processor -- start 2023-11-28T11:09:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 -- start start 2023-11-28T11:09:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcc8c0cb650 0x7fcc8c153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c1f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc8c158cd0 0x7fcc8c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c0ad3e0 con 0x7fcc8c158900 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c1556d0 con 0x7fcc8c158cd0 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc93d42700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c1f60f0 con 0x7fcc8c0cb650 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc912dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c1f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc912dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c1f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49502/0 (socket says 172.21.15.83:49502) 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc912dd700 1 -- 172.21.15.83:0/673550080 learned_addr learned my addr 172.21.15.83:0/673550080 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc922df700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc8c158cd0 0x7fcc8c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.953+0000 7fcc91ade700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcc8c0cb650 0x7fcc8c153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc912dd700 1 -- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcc8c0cb650 msgr2=0x7fcc8c153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc912dd700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcc8c0cb650 0x7fcc8c153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc912dd700 1 -- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc8c158cd0 msgr2=0x7fcc8c155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc912dd700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc8c158cd0 0x7fcc8c155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc912dd700 1 -- 172.21.15.83:0/673550080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c00b740 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc912dd700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c1f5bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800c820 tx=0x7fcc8800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88004220 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88004380 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc880036a0 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c15cd50 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.954+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c15ca70 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.955+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c15d030 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.955+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc88020070 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.956+0000 7fcc827fc700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcc700682f0 0x7fcc7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.956+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcc88085950 con 0x7fcc8c158900 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.956+0000 7fcc91ade700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcc700682f0 0x7fcc7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.956+0000 7fcc91ade700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcc700682f0 0x7fcc7006a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b2c0 tx=0x7fcc7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:12.960+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcc88051ec0 con 0x7fcc8c158900 2023-11-28T11:09:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.143+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc8c0742b0 con 0x7fcc8c158900 2023-11-28T11:09:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.143+0000 7fcc827fc700 1 -- 172.21.15.83:0/673550080 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc880554e0 con 0x7fcc8c158900 2023-11-28T11:09:13.145 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcc700682f0 msgr2=0x7fcc7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcc700682f0 0x7fcc7006a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b2c0 tx=0x7fcc7c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 msgr2=0x7fcc8c1f5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c1f5bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800c820 tx=0x7fcc8800cb30 comp rx=0 tx=0).stop 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 shutdown_connections 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc8c158cd0 0x7fcc8c155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcc8c0cb650 0x7fcc8c153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcc700682f0 0x7fcc7006a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 --2- 172.21.15.83:0/673550080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc8c158900 0x7fcc8c1f5bb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 >> 172.21.15.83:0/673550080 conn(0x7fcc8c0b1090 msgr2=0x7fcc8c157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 shutdown_connections 2023-11-28T11:09:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.145+0000 7fcc93d42700 1 -- 172.21.15.83:0/673550080 wait complete. 2023-11-28T11:09:13.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:13.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.281+0000 7fe31746f700 1 Processor -- start 2023-11-28T11:09:13.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.281+0000 7fe31746f700 1 -- start start 2023-11-28T11:09:13.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31746f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe3100cd670 0x7fe310071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31746f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 0x7fe31006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31746f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3100cdb10 con 0x7fe310158c50 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31746f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100cab20 con 0x7fe3100cd670 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31746f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe310159020 con 0x7fe3100718a0 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31520b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe310158c50 0x7fe3100c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47368/0 (socket says 172.21.15.83:47368) 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe31520b700 1 -- 172.21.15.83:0/760198873 learned_addr learned my addr 172.21.15.83:0/760198873 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe314a0a700 1 --2- 172.21.15.83:0/760198873 >> v2:172.21.15.90:3300/0 conn(0x7fe3100cd670 0x7fe310071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202844843 0 0) 0x7fe3100cdb10 con 0x7fe310158c50 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe315a0c700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 0x7fe31006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.282+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2fc0057c0 con 0x7fe310158c50 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952036619 0 0) 0x7fe2fc0057c0 con 0x7fe310158c50 2023-11-28T11:09:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 msgr2=0x7fe31006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 0x7fe31006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 >> v2:172.21.15.90:3300/0 conn(0x7fe3100cd670 msgr2=0x7fe310071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 --2- 172.21.15.83:0/760198873 >> v2:172.21.15.90:3300/0 conn(0x7fe3100cd670 0x7fe310071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100c9ea0 con 0x7fe310158c50 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fe300001670 con 0x7fe310158c50 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 >> v1:172.21.15.83:6789/0 conn(0x7fe310158c50 legacy=0x7fe3100c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc005c40 0x7fe2fc006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc006a60 0x7fe2fc006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 0x7fe2fc007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100cab20 con 0x7fe2fc006a60 2023-11-28T11:09:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc00dc20 con 0x7fe2fc005c40 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc00dd60 con 0x7fe3100718a0 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc006a60 0x7fe2fc006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe31520b700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc005c40 0x7fe2fc006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe315a0c700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 0x7fe2fc007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 -- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 msgr2=0x7fe2fc007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3100718a0 0x7fe2fc007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 -- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc005c40 msgr2=0x7fe2fc006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc005c40 0x7fe2fc006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 -- 172.21.15.83:0/760198873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc0057c0 con 0x7fe2fc006a60 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.283+0000 7fe314a0a700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc006a60 0x7fe2fc006ed0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009b60 tx=0x7fe2fc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f00af3750186fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00e6c0 con 0x7fe2fc006a60 2023-11-28T11:09:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c00e820 con 0x7fe2fc006a60 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe307fff700 1 -- 172.21.15.83:0/760198873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00eb90 con 0x7fe2fc006a60 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe31746f700 1 -- 172.21.15.83:0/760198873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc006a60 msgr2=0x7fe2fc006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe31746f700 1 --2- 172.21.15.83:0/760198873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc006a60 0x7fe2fc006ed0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009b60 tx=0x7fe2fc007a90 comp rx=0 tx=0).stop 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe315a0c700 1 -- 172.21.15.83:0/760198873 reap_dead start 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe31746f700 1 -- 172.21.15.83:0/760198873 shutdown_connections 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe31746f700 1 -- 172.21.15.83:0/760198873 >> 172.21.15.83:0/760198873 conn(0x7fe3100ceb80 msgr2=0x7fe3100cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe31746f700 1 -- 172.21.15.83:0/760198873 shutdown_connections 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.284+0000 7fe31746f700 1 -- 172.21.15.83:0/760198873 wait complete. 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.285+0000 7fe31746f700 1 Processor -- start 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.285+0000 7fe31746f700 1 -- start start 2023-11-28T11:09:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31746f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc006a60 0x7fe3101522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31746f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 0x7fe310159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31746f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3101f19e0 0x7fe3101f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31746f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100a74a0 con 0x7fe3101594e0 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31746f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100b38a0 con 0x7fe2fc006a60 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31746f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe310159e70 con 0x7fe3101f19e0 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 0x7fe310159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 0x7fe310159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49522/0 (socket says 172.21.15.83:49522) 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 -- 172.21.15.83:0/4121637970 learned_addr learned my addr 172.21.15.83:0/4121637970 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe315a0c700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3101f19e0 0x7fe3101f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe31520b700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc006a60 0x7fe3101522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 -- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3101f19e0 msgr2=0x7fe3101f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3101f19e0 0x7fe3101f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 -- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc006a60 msgr2=0x7fe3101522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc006a60 0x7fe3101522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.286+0000 7fe314a0a700 1 -- 172.21.15.83:0/4121637970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100ca460 con 0x7fe3101594e0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe31520b700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc006a60 0x7fe3101522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe314a0a700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 0x7fe310159930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0068b0 tx=0x7fe30c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00e6c0 con 0x7fe3101594e0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c003ab0 con 0x7fe3101594e0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c003e60 con 0x7fe3101594e0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3100ca180 con 0x7fe3101594e0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.287+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3100ca740 con 0x7fe3101594e0 2023-11-28T11:09:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.288+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3100742b0 con 0x7fe3101594e0 2023-11-28T11:09:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.289+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe30c019030 con 0x7fe3101594e0 2023-11-28T11:09:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.289+0000 7fe305ffb700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe2f40682f0 0x7fe2f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.289+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe30c0876e0 con 0x7fe3101594e0 2023-11-28T11:09:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.289+0000 7fe31520b700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe2f40682f0 0x7fe2f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.289+0000 7fe31520b700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe2f40682f0 0x7fe2f406a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe300007cf0 tx=0x7fe300009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.293+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe30c053c50 con 0x7fe3101594e0 2023-11-28T11:09:13.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.475+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3100b0460 con 0x7fe3101594e0 2023-11-28T11:09:13.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.475+0000 7fe305ffb700 1 -- 172.21.15.83:0/4121637970 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe30c057270 con 0x7fe3101594e0 2023-11-28T11:09:13.476 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:13.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe2f40682f0 msgr2=0x7fe2f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe2f40682f0 0x7fe2f406a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe300007cf0 tx=0x7fe300009040 comp rx=0 tx=0).stop 2023-11-28T11:09:13.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 msgr2=0x7fe310159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 0x7fe310159930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0068b0 tx=0x7fe30c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 shutdown_connections 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3101f19e0 0x7fe3101f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2fc006a60 0x7fe3101522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe2f40682f0 0x7fe2f406a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 --2- 172.21.15.83:0/4121637970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3101594e0 0x7fe310159930 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 >> 172.21.15.83:0/4121637970 conn(0x7fe3100ceb80 msgr2=0x7fe3100cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 shutdown_connections 2023-11-28T11:09:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.477+0000 7fe31746f700 1 -- 172.21.15.83:0/4121637970 wait complete. 2023-11-28T11:09:13.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:13.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.612+0000 7f36ccb45700 1 Processor -- start 2023-11-28T11:09:13.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.612+0000 7f36ccb45700 1 -- start start 2023-11-28T11:09:13.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36ccb45700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f36c80aee20 0x7f36c80b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36ccb45700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c80b5b00 con 0x7f36c806c370 2023-11-28T11:09:13.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36ccb45700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c806c800 con 0x7f36c80af1f0 2023-11-28T11:09:13.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36ccb45700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c806cae0 con 0x7f36c80aee20 2023-11-28T11:09:13.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c5d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f36c806c370 0x7f36c806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47390/0 (socket says 172.21.15.83:47390) 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c5d9b700 1 -- 172.21.15.83:0/1070170035 learned_addr learned my addr 172.21.15.83:0/1070170035 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c659c700 1 --2- 172.21.15.83:0/1070170035 >> v2:172.21.15.90:3300/0 conn(0x7f36c80aee20 0x7f36c80b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732507946 0 0) 0x7f36b8001110 con 0x7f36c806c370 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b00035b0 con 0x7f36c806c370 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836488038 0 0) 0x7f36b00035b0 con 0x7f36c806c370 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 >> v2:172.21.15.90:3300/0 conn(0x7f36c80aee20 msgr2=0x7f36c80b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 --2- 172.21.15.83:0/1070170035 >> v2:172.21.15.90:3300/0 conn(0x7f36c80aee20 0x7f36c80b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 >> v1:172.21.15.90:6789/0 conn(0x7f36c80af1f0 legacy=0x7f36c80cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c806cdc0 con 0x7f36c806c370 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f36b8001800 con 0x7f36c806c370 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 >> v1:172.21.15.83:6789/0 conn(0x7f36c806c370 legacy=0x7f36c806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0003a00 0x7f36b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36b0004850 0x7f36b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36b0005200 0x7f36b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c806cae0 con 0x7f36b0005200 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b000c110 con 0x7f36b0003a00 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b000c2d0 con 0x7f36b0004850 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36b0005200 0x7f36b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c659c700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0003a00 0x7f36b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.614+0000 7f36c5d9b700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36b0004850 0x7f36b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 -- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36b0004850 msgr2=0x7f36b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36b0004850 0x7f36b0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 -- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0003a00 msgr2=0x7f36b0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0003a00 0x7f36b0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 -- 172.21.15.83:0/1070170035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00035b0 con 0x7f36b0005200 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 -- 172.21.15.83:0/1070170035 reap_dead start 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36b0005200 0x7f36b000ba10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36c000c190 tx=0x7f36b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b71e20c55997392 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0001db0 con 0x7f36b0005200 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c000ccd0 con 0x7f36b0005200 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36c559a700 1 -- 172.21.15.83:0/1070170035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0003c40 con 0x7f36b0005200 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36ccb45700 1 -- 172.21.15.83:0/1070170035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36b0005200 msgr2=0x7f36b000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36ccb45700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36b0005200 0x7f36b000ba10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36c000c190 tx=0x7f36b0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36ccb45700 1 -- 172.21.15.83:0/1070170035 shutdown_connections 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36ccb45700 1 --2- 172.21.15.83:0/1070170035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36b0005200 0x7f36b000ba10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.615+0000 7f36ccb45700 1 -- 172.21.15.83:0/1070170035 >> 172.21.15.83:0/1070170035 conn(0x7f36c80b2b60 msgr2=0x7f36c80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.616+0000 7f36ccb45700 1 -- 172.21.15.83:0/1070170035 shutdown_connections 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.616+0000 7f36ccb45700 1 -- 172.21.15.83:0/1070170035 wait complete. 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.617+0000 7f36ccb45700 1 Processor -- start 2023-11-28T11:09:13.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.617+0000 7f36ccb45700 1 -- start start 2023-11-28T11:09:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0005200 0x7f36c80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c80b0fb0 0x7f36c8155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 0x7f36c8155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c815b730 con 0x7f36c8155a40 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80b0960 con 0x7f36b0005200 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81563f0 con 0x7f36c80b0fb0 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 0x7f36c8155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 0x7f36c8155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49544/0 (socket says 172.21.15.83:49544) 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 -- 172.21.15.83:0/642520855 learned_addr learned my addr 172.21.15.83:0/642520855 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c659c700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0005200 0x7f36c80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c5d9b700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c80b0fb0 0x7f36c8155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 -- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c80b0fb0 msgr2=0x7f36c8155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c80b0fb0 0x7f36c8155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 -- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0005200 msgr2=0x7f36c80cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0005200 0x7f36c80cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 -- 172.21.15.83:0/642520855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c806fc70 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36c6d9d700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 0x7f36c8155eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36c80b3b80 tx=0x7f36c00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0005070 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c00051d0 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c806d0a0 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0005360 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.618+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c806d380 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.620+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c80cbd50 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.620+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f36c0019040 con 0x7f36c8155a40 2023-11-28T11:09:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.620+0000 7f36b6ffd700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36a80682f0 0x7f36a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.620+0000 7f36c659c700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36a80682f0 0x7f36a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.620+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f36c0086830 con 0x7f36c8155a40 2023-11-28T11:09:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.621+0000 7f36c659c700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36a80682f0 0x7f36a806a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009bc0 tx=0x7f36bc0091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.624+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f36c0052da0 con 0x7f36c8155a40 2023-11-28T11:09:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.807+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36c815b870 con 0x7f36c8155a40 2023-11-28T11:09:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.807+0000 7f36b6ffd700 1 -- 172.21.15.83:0/642520855 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f36c815b870 con 0x7f36c8155a40 2023-11-28T11:09:13.808 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36a80682f0 msgr2=0x7f36a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36a80682f0 0x7f36a806a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009bc0 tx=0x7f36bc0091c0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 msgr2=0x7f36c8155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 0x7f36c8155eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36c80b3b80 tx=0x7f36c00014a0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 shutdown_connections 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f36a80682f0 0x7f36a806a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36c8155a40 0x7f36c8155eb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36b0005200 0x7f36c80cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 --2- 172.21.15.83:0/642520855 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f36c80b0fb0 0x7f36c8155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 >> 172.21.15.83:0/642520855 conn(0x7f36c80b2b60 msgr2=0x7f36c80cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 shutdown_connections 2023-11-28T11:09:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.809+0000 7f36ccb45700 1 -- 172.21.15.83:0/642520855 wait complete. 2023-11-28T11:09:13.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:13.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.941+0000 7f44c96e7700 1 Processor -- start 2023-11-28T11:09:13.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.941+0000 7f44c96e7700 1 -- start start 2023-11-28T11:09:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c96e7700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f44c415a510 0x7f44c40cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c96e7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44c415a8e0 0x7f44c40b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c96e7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c415f1d0 con 0x7f44c40cfdf0 2023-11-28T11:09:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c96e7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c40cd780 con 0x7f44c415a8e0 2023-11-28T11:09:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c96e7700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c415eef0 con 0x7f44c415a510 2023-11-28T11:09:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44c415a8e0 0x7f44c40b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44c415a8e0 0x7f44c40b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49550/0 (socket says 172.21.15.83:49550) 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c27fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f44c40cfdf0 0x7f44c40b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47408/0 (socket says 172.21.15.83:47408) 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c37fe700 1 -- 172.21.15.83:0/2410273184 learned_addr learned my addr 172.21.15.83:0/2410273184 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c2ffd700 1 --2- 172.21.15.83:0/2410273184 >> v2:172.21.15.90:3300/0 conn(0x7f44c415a510 0x7f44c40cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c37fe700 1 -- 172.21.15.83:0/2410273184 >> v2:172.21.15.90:3300/0 conn(0x7f44c415a510 msgr2=0x7f44c40cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.942+0000 7f44c37fe700 1 --2- 172.21.15.83:0/2410273184 >> v2:172.21.15.90:3300/0 conn(0x7f44c415a510 0x7f44c40cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c37fe700 1 -- 172.21.15.83:0/2410273184 >> v1:172.21.15.83:6789/0 conn(0x7f44c40cfdf0 legacy=0x7f44c40b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c37fe700 1 -- 172.21.15.83:0/2410273184 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40cedd0 con 0x7f44c415a8e0 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c1ffb700 1 -- 172.21.15.83:0/2410273184 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405710747 0 0) 0x7f44c415f1d0 con 0x7f44c40cfdf0 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c37fe700 1 --2- 172.21.15.83:0/2410273184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c415a8e0 0x7f44c40b3d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ce6b0 tx=0x7f44bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c1aa3c661a3b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c1ffb700 1 -- 172.21.15.83:0/2410273184 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00e070 con 0x7f44c415a8e0 2023-11-28T11:09:13.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c1ffb700 1 -- 172.21.15.83:0/2410273184 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc00b9c0 con 0x7f44c415a8e0 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c1ffb700 1 -- 172.21.15.83:0/2410273184 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00bd70 con 0x7f44c415a8e0 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 -- 172.21.15.83:0/2410273184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c415a8e0 msgr2=0x7f44c40b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2410273184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c415a8e0 0x7f44c40b3d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ce6b0 tx=0x7f44bc00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 -- 172.21.15.83:0/2410273184 shutdown_connections 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2410273184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c415a8e0 0x7f44c40b3d30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2410273184 >> v2:172.21.15.90:3300/0 conn(0x7f44c415a510 0x7f44c40cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 -- 172.21.15.83:0/2410273184 >> 172.21.15.83:0/2410273184 conn(0x7f44c40cd960 msgr2=0x7f44c40cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 -- 172.21.15.83:0/2410273184 shutdown_connections 2023-11-28T11:09:13.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.943+0000 7f44c96e7700 1 -- 172.21.15.83:0/2410273184 wait complete. 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.945+0000 7f44c96e7700 1 Processor -- start 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.945+0000 7f44c96e7700 1 -- start start 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c96e7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 0x7f44c4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c96e7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c415a510 0x7f44c415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c96e7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44c415a8e0 0x7f44c415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c96e7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c415c6a0 con 0x7f44c40cfdf0 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c96e7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c4158200 con 0x7f44c415a510 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c96e7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40c9f10 con 0x7f44c415a8e0 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 0x7f44c4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 0x7f44c4157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49552/0 (socket says 172.21.15.83:49552) 2023-11-28T11:09:13.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c2ffd700 1 -- 172.21.15.83:0/2712611075 learned_addr learned my addr 172.21.15.83:0/2712611075 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c27fc700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c415a510 0x7f44c415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c37fe700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44c415a8e0 0x7f44c415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c2ffd700 1 -- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44c415a8e0 msgr2=0x7f44c415acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.946+0000 7f44c2ffd700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44c415a8e0 0x7f44c415acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c2ffd700 1 -- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c415a510 msgr2=0x7f44c415cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c2ffd700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c415a510 0x7f44c415cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c2ffd700 1 -- 172.21.15.83:0/2712611075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc009b30 con 0x7f44c40cfdf0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c37fe700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44c415a8e0 0x7f44c415acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c2ffd700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 0x7f44c4157cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44b800e810 tx=0x7f44b800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8013610 con 0x7f44c40cfdf0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b8013770 con 0x7f44c40cfdf0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c40cf390 con 0x7f44c40cfdf0 2023-11-28T11:09:13.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8013b20 con 0x7f44c40cfdf0 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.947+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c40cf0b0 con 0x7f44c40cfdf0 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.948+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c40b3070 con 0x7f44c40cfdf0 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.948+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f44b8023070 con 0x7f44c40cfdf0 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.949+0000 7f44ab7fe700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f44b00682f0 0x7f44b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.949+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f44b8085c60 con 0x7f44c40cfdf0 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.949+0000 7f44c27fc700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f44b00682f0 0x7f44b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.949+0000 7f44c27fc700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f44b00682f0 0x7f44b006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f44c40c4980 tx=0x7f44b4007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:13.953+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f44b8052150 con 0x7f44c40cfdf0 2023-11-28T11:09:14.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.138+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44c40c86a0 con 0x7f44c40cfdf0 2023-11-28T11:09:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.139+0000 7f44ab7fe700 1 -- 172.21.15.83:0/2712611075 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f44b8055770 con 0x7f44c40cfdf0 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f44b00682f0 msgr2=0x7f44b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f44b00682f0 0x7f44b006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f44c40c4980 tx=0x7f44b4007e70 comp rx=0 tx=0).stop 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 msgr2=0x7f44c4157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 0x7f44c4157cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44b800e810 tx=0x7f44b800eb20 comp rx=0 tx=0).stop 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 shutdown_connections 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44c415a8e0 0x7f44c415acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f44b00682f0 0x7f44b006a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c40cfdf0 0x7f44c4157cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 --2- 172.21.15.83:0/2712611075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c415a510 0x7f44c415cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.141+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 >> 172.21.15.83:0/2712611075 conn(0x7f44c40cd960 msgr2=0x7f44c40b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.142+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 shutdown_connections 2023-11-28T11:09:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.142+0000 7f44c96e7700 1 -- 172.21.15.83:0/2712611075 wait complete. 2023-11-28T11:09:14.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:14.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:14.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.277+0000 7f8070076700 1 Processor -- start 2023-11-28T11:09:14.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.277+0000 7f8070076700 1 -- start start 2023-11-28T11:09:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f8070076700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 0x7f80680aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f8070076700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f806806e370 0x7f806806b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f8070076700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806806d4a0 con 0x7f80680b4bb0 2023-11-28T11:09:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f8070076700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680b5020 con 0x7f806806e370 2023-11-28T11:09:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f8070076700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b5ae0 con 0x7f806815c890 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f806d611700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f80680b4bb0 0x7f806806dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47420/0 (socket says 172.21.15.83:47420) 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f806d611700 1 -- 172.21.15.83:0/2729686331 learned_addr learned my addr 172.21.15.83:0/2729686331 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f806e613700 1 --2- 172.21.15.83:0/2729686331 >> v2:172.21.15.90:3300/0 conn(0x7f806806e370 0x7f806806b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555755508 0 0) 0x7f806806d4a0 con 0x7f80680b4bb0 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f806de12700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 0x7f80680aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.278+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80540035b0 con 0x7f80680b4bb0 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414176897 0 0) 0x7f80540035b0 con 0x7f80680b4bb0 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 msgr2=0x7f80680aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 0x7f80680aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 >> v2:172.21.15.90:3300/0 conn(0x7f806806e370 msgr2=0x7f806806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 --2- 172.21.15.83:0/2729686331 >> v2:172.21.15.90:3300/0 conn(0x7f806806e370 0x7f806806b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806806c400 con 0x7f80680b4bb0 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f8064001670 con 0x7f80680b4bb0 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 >> v1:172.21.15.83:6789/0 conn(0x7f80680b4bb0 legacy=0x7f806806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f8054003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8054004430 0x7f80540048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 0x7f8054005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b5020 con 0x7f8054003a00 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b5ae0 con 0x7f8054004430 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805400ba10 con 0x7f806815c890 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f8054003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806e613700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 0x7f8054005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806d611700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8054004430 0x7f80540048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 -- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 msgr2=0x7f8054005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f806815c890 0x7f8054005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 -- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8054004430 msgr2=0x7f80540048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8054004430 0x7f80540048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 -- 172.21.15.83:0/2729686331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80540035b0 con 0x7f8054003a00 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806de12700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f8054003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8058009b70 tx=0x7f8054005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae044b87ffe72d00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058005a20 con 0x7f8054003a00 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8058005b80 con 0x7f8054003a00 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.279+0000 7f806ce10700 1 -- 172.21.15.83:0/2729686331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805800e6c0 con 0x7f8054003a00 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f8070076700 1 -- 172.21.15.83:0/2729686331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 msgr2=0x7f8054003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f8070076700 1 --2- 172.21.15.83:0/2729686331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f8054003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8058009b70 tx=0x7f8054005880 comp rx=0 tx=0).stop 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f806e613700 1 -- 172.21.15.83:0/2729686331 reap_dead start 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f8070076700 1 -- 172.21.15.83:0/2729686331 shutdown_connections 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f8070076700 1 -- 172.21.15.83:0/2729686331 >> 172.21.15.83:0/2729686331 conn(0x7f80680b2b60 msgr2=0x7f80680b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f8070076700 1 -- 172.21.15.83:0/2729686331 shutdown_connections 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.280+0000 7f8070076700 1 -- 172.21.15.83:0/2729686331 wait complete. 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.281+0000 7f8070076700 1 Processor -- start 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.281+0000 7f8070076700 1 -- start start 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f8070076700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f80680cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f8070076700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80680c5e70 0x7f80681524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f8070076700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80680c62e0 0x7f8068152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f8070076700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680c38c0 con 0x7f8054003a00 2023-11-28T11:09:14.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f8070076700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680c3ba0 con 0x7f80680c5e70 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f8070076700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80680c3e80 con 0x7f80680c62e0 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f806de12700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f80680cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f806de12700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f80680cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49570/0 (socket says 172.21.15.83:49570) 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f806de12700 1 -- 172.21.15.83:0/724552101 learned_addr learned my addr 172.21.15.83:0/724552101 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f806d611700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80680c5e70 0x7f80681524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.282+0000 7f806e613700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80680c62e0 0x7f8068152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806de12700 1 -- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80680c62e0 msgr2=0x7f8068152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806de12700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80680c62e0 0x7f8068152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806de12700 1 -- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80680c5e70 msgr2=0x7f80681524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806de12700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80680c5e70 0x7f80681524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806de12700 1 -- 172.21.15.83:0/724552101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680b5520 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806d611700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80680c5e70 0x7f80681524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f806de12700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f80680cfdb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8054005770 tx=0x7f80580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058020070 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8058005e30 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058004480 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80680b5240 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.283+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80680b5800 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.284+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80680b3820 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.284+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f805802b020 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.285+0000 7f805e7fc700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f804c0682f0 0x7f804c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.285+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f80580872c0 con 0x7f8054003a00 2023-11-28T11:09:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.285+0000 7f806d611700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f804c0682f0 0x7f804c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.285+0000 7f806d611700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f804c0682f0 0x7f804c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8064000fb0 tx=0x7f8064009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.289+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8058053830 con 0x7f8054003a00 2023-11-28T11:09:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.470+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f80680742b0 con 0x7f8054003a00 2023-11-28T11:09:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.470+0000 7f805e7fc700 1 -- 172.21.15.83:0/724552101 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8058056e50 con 0x7f8054003a00 2023-11-28T11:09:14.471 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f804c0682f0 msgr2=0x7f804c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f804c0682f0 0x7f804c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8064000fb0 tx=0x7f8064009040 comp rx=0 tx=0).stop 2023-11-28T11:09:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 msgr2=0x7f80680cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f80680cfdb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8054005770 tx=0x7f80580047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 shutdown_connections 2023-11-28T11:09:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80680c62e0 0x7f8068152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f804c0682f0 0x7f804c06a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8054003a00 0x7f80680cfdb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 --2- 172.21.15.83:0/724552101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80680c5e70 0x7f80681524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 >> 172.21.15.83:0/724552101 conn(0x7f80680b2b60 msgr2=0x7f80680ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 shutdown_connections 2023-11-28T11:09:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.472+0000 7f8070076700 1 -- 172.21.15.83:0/724552101 wait complete. 2023-11-28T11:09:14.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.608+0000 7f33ac801700 1 Processor -- start 2023-11-28T11:09:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.608+0000 7f33ac801700 1 -- start start 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33ac801700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f33a815c390 0x7f33a80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33ac801700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33a80ca520 0x7f33a80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33ac801700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a80cd920 con 0x7f33a80ccdf0 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33ac801700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a80ca970 con 0x7f33a80ca520 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33ac801700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a80cd220 con 0x7f33a815c390 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33a80ca520 0x7f33a80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33a80ca520 0x7f33a80cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49578/0 (socket says 172.21.15.83:49578) 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 -- 172.21.15.83:0/3178115507 learned_addr learned my addr 172.21.15.83:0/3178115507 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 -- 172.21.15.83:0/3178115507 >> v2:172.21.15.90:3300/0 conn(0x7f33a815c390 msgr2=0x7f33a80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 --2- 172.21.15.83:0/3178115507 >> v2:172.21.15.90:3300/0 conn(0x7f33a815c390 0x7f33a80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 -- 172.21.15.83:0/3178115507 >> v1:172.21.15.90:6789/0 conn(0x7f33a80ccdf0 legacy=0x7f33a815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 -- 172.21.15.83:0/3178115507 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a806afc0 con 0x7f33a80ca520 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a659c700 1 --2- 172.21.15.83:0/3178115507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ca520 0x7f33a80cc8b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f33a80cd680 tx=0x7f3398009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3abad240978e3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a5d9b700 1 -- 172.21.15.83:0/3178115507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398004230 con 0x7f33a80ca520 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a5d9b700 1 -- 172.21.15.83:0/3178115507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398004390 con 0x7f33a80ca520 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.609+0000 7f33a5d9b700 1 -- 172.21.15.83:0/3178115507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980036a0 con 0x7f33a80ca520 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 -- 172.21.15.83:0/3178115507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ca520 msgr2=0x7f33a80cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 --2- 172.21.15.83:0/3178115507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ca520 0x7f33a80cc8b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f33a80cd680 tx=0x7f3398009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 -- 172.21.15.83:0/3178115507 shutdown_connections 2023-11-28T11:09:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 --2- 172.21.15.83:0/3178115507 >> v2:172.21.15.90:3300/0 conn(0x7f33a815c390 0x7f33a80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 --2- 172.21.15.83:0/3178115507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ca520 0x7f33a80cc8b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 -- 172.21.15.83:0/3178115507 >> 172.21.15.83:0/3178115507 conn(0x7f33a806de00 msgr2=0x7f33a806e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 -- 172.21.15.83:0/3178115507 shutdown_connections 2023-11-28T11:09:14.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.610+0000 7f33ac801700 1 -- 172.21.15.83:0/3178115507 wait complete. 2023-11-28T11:09:14.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.611+0000 7f33ac801700 1 Processor -- start 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.611+0000 7f33ac801700 1 -- start start 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33ac801700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a80ca520 0x7f33a814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33ac801700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 0x7f33a80ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33ac801700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f33a815c390 0x7f33a81f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33ac801700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80acf10 con 0x7f33a80ccdf0 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33ac801700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80ad1f0 con 0x7f33a80ca520 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33ac801700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80ad4d0 con 0x7f33a815c390 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33a659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 0x7f33a80ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33a659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 0x7f33a80ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49586/0 (socket says 172.21.15.83:49586) 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.612+0000 7f33a659c700 1 -- 172.21.15.83:0/752755565 learned_addr learned my addr 172.21.15.83:0/752755565 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a6d9d700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a80ca520 0x7f33a814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a759e700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f33a815c390 0x7f33a81f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a659c700 1 -- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f33a815c390 msgr2=0x7f33a81f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a659c700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f33a815c390 0x7f33a81f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a659c700 1 -- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a80ca520 msgr2=0x7f33a814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a659c700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a80ca520 0x7f33a814acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a659c700 1 -- 172.21.15.83:0/752755565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398009550 con 0x7f33a80ccdf0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a6d9d700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a80ca520 0x7f33a814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33a659c700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 0x7f33a80ac290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f33a80b1700 tx=0x7f3398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398003dc0 con 0x7f33a80ccdf0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339801f070 con 0x7f33a80ccdf0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398011870 con 0x7f33a80ccdf0 2023-11-28T11:09:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a80cae30 con 0x7f33a80ccdf0 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.613+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a80cab50 con 0x7f33a80ccdf0 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.614+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a80cb110 con 0x7f33a80ccdf0 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.614+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3398017020 con 0x7f33a80ccdf0 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.615+0000 7f338f7fe700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f33940682f0 0x7f339406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.615+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3398087820 con 0x7f33a80ccdf0 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.615+0000 7f33a6d9d700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f33940682f0 0x7f339406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.615+0000 7f33a6d9d700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f33940682f0 0x7f339406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f339c00a8f0 tx=0x7f339c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.619+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3398053d30 con 0x7f33a80ccdf0 2023-11-28T11:09:14.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.797+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33a8074240 con 0x7f33a80ccdf0 2023-11-28T11:09:14.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.797+0000 7f338f7fe700 1 -- 172.21.15.83:0/752755565 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3398057350 con 0x7f33a80ccdf0 2023-11-28T11:09:14.798 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:14.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f33940682f0 msgr2=0x7f339406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f33940682f0 0x7f339406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f339c00a8f0 tx=0x7f339c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:14.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 msgr2=0x7f33a80ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 0x7f33a80ac290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f33a80b1700 tx=0x7f3398004750 comp rx=0 tx=0).stop 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 shutdown_connections 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f33a815c390 0x7f33a81f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a80ca520 0x7f33a814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f33940682f0 0x7f339406a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 --2- 172.21.15.83:0/752755565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a80ccdf0 0x7f33a80ac290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 >> 172.21.15.83:0/752755565 conn(0x7f33a806de00 msgr2=0x7f33a806c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 shutdown_connections 2023-11-28T11:09:14.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.799+0000 7f33ac801700 1 -- 172.21.15.83:0/752755565 wait complete. 2023-11-28T11:09:14.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.936+0000 7fae1c993700 1 Processor -- start 2023-11-28T11:09:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.936+0000 7fae1c993700 1 -- start start 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.936+0000 7fae1c993700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fae180c9bc0 0x7fae1806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1c993700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1815e4f0 con 0x7fae1815dff0 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1c993700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1815e7d0 con 0x7fae180cc020 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1c993700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae180cbc80 con 0x7fae180c9bc0 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae15d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fae1815dff0 0x7fae180cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47452/0 (socket says 172.21.15.83:47452) 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae15d9b700 1 -- 172.21.15.83:0/2037723380 learned_addr learned my addr 172.21.15.83:0/2037723380 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849307844 0 0) 0x7fae1815e4f0 con 0x7fae1815dff0 2023-11-28T11:09:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae000035b0 con 0x7fae1815dff0 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae16d9d700 1 --2- 172.21.15.83:0/2037723380 >> v2:172.21.15.90:3300/0 conn(0x7fae180c9bc0 0x7fae1806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132609432 0 0) 0x7fae000035b0 con 0x7fae1815dff0 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 >> v2:172.21.15.90:3300/0 conn(0x7fae180c9bc0 msgr2=0x7fae1806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 --2- 172.21.15.83:0/2037723380 >> v2:172.21.15.90:3300/0 conn(0x7fae180c9bc0 0x7fae1806a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 >> v1:172.21.15.90:6789/0 conn(0x7fae180cc020 legacy=0x7fae180cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1815eab0 con 0x7fae1815dff0 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fae0c001670 con 0x7fae1815dff0 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae16d9d700 1 --2- 172.21.15.83:0/2037723380 >> v2:172.21.15.90:3300/0 conn(0x7fae180c9bc0 0x7fae1806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 >> v1:172.21.15.83:6789/0 conn(0x7fae1815dff0 legacy=0x7fae180cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae00003a00 0x7fae00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00004850 0x7fae00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00005400 0x7fae0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180cbc80 con 0x7fae00005400 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c280 con 0x7fae00004850 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c440 con 0x7fae00003a00 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae16d9d700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00005400 0x7fae0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae1659c700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae00003a00 0x7fae00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.937+0000 7fae15d9b700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00004850 0x7fae00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 -- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae00003a00 msgr2=0x7fae00003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae00003a00 0x7fae00003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 -- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00004850 msgr2=0x7fae00004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00004850 0x7fae00004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 -- 172.21.15.83:0/2037723380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000035b0 con 0x7fae00005400 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 -- 172.21.15.83:0/2037723380 reap_dead start 2023-11-28T11:09:14.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae16d9d700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00005400 0x7fae0000ba10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fae100073e0 tx=0x7fae00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88c660090ca77b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10009490 con 0x7fae00005400 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae10014400 con 0x7fae00005400 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1559a700 1 -- 172.21.15.83:0/2037723380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae100147b0 con 0x7fae00005400 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1c993700 1 -- 172.21.15.83:0/2037723380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00005400 msgr2=0x7fae0000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1c993700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00005400 0x7fae0000ba10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fae100073e0 tx=0x7fae00005880 comp rx=0 tx=0).stop 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1c993700 1 -- 172.21.15.83:0/2037723380 shutdown_connections 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.938+0000 7fae1c993700 1 --2- 172.21.15.83:0/2037723380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00005400 0x7fae0000ba10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.939+0000 7fae1c993700 1 -- 172.21.15.83:0/2037723380 >> 172.21.15.83:0/2037723380 conn(0x7fae180b1090 msgr2=0x7fae180b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.939+0000 7fae1c993700 1 -- 172.21.15.83:0/2037723380 shutdown_connections 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.939+0000 7fae1c993700 1 -- 172.21.15.83:0/2037723380 wait complete. 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.940+0000 7fae1c993700 1 Processor -- start 2023-11-28T11:09:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.940+0000 7fae1c993700 1 -- start start 2023-11-28T11:09:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00005400 0x7fae180c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae18149600 0x7fae180b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 0x7fae18148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c34a0 con 0x7fae180b42d0 2023-11-28T11:09:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c2e70 con 0x7fae00005400 2023-11-28T11:09:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1815af20 con 0x7fae18149600 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 0x7fae18148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 0x7fae18148940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49608/0 (socket says 172.21.15.83:49608) 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 -- 172.21.15.83:0/1718018856 learned_addr learned my addr 172.21.15.83:0/1718018856 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1659c700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00005400 0x7fae180c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae15d9b700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae18149600 0x7fae180b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 -- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae18149600 msgr2=0x7fae180b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae18149600 0x7fae180b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 -- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00005400 msgr2=0x7fae180c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00005400 0x7fae180c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 -- 172.21.15.83:0/1718018856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1815aa00 con 0x7fae180b42d0 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae15d9b700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae18149600 0x7fae180b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1659c700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00005400 0x7fae180c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae16d9d700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 0x7fae18148940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae1806b310 tx=0x7fae10004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10007000 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae10009450 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10003e10 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1815ed90 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.941+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae180c9fe0 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.942+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae180c35e0 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.943+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae10012070 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.943+0000 7fae06ffd700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fadf80682f0 0x7fadf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.943+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fae10087460 con 0x7fae180b42d0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.943+0000 7fae1659c700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fadf80682f0 0x7fadf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.944+0000 7fae1659c700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fadf80682f0 0x7fadf806a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae08000c00 tx=0x7fae08009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:14.948+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae180c35e0 con 0x7fae180b42d0 2023-11-28T11:09:15.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.130+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae180c35e0 con 0x7fae180b42d0 2023-11-28T11:09:15.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.130+0000 7fae06ffd700 1 -- 172.21.15.83:0/1718018856 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae180c35e0 con 0x7fae180b42d0 2023-11-28T11:09:15.131 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fadf80682f0 msgr2=0x7fadf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fadf80682f0 0x7fadf806a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae08000c00 tx=0x7fae08009040 comp rx=0 tx=0).stop 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 msgr2=0x7fae18148940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 0x7fae18148940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae1806b310 tx=0x7fae10004b40 comp rx=0 tx=0).stop 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 shutdown_connections 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fadf80682f0 0x7fadf806a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae180b42d0 0x7fae18148940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae00005400 0x7fae180c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 --2- 172.21.15.83:0/1718018856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae18149600 0x7fae180b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 >> 172.21.15.83:0/1718018856 conn(0x7fae180b1090 msgr2=0x7fae18154850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 shutdown_connections 2023-11-28T11:09:15.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.132+0000 7fae1c993700 1 -- 172.21.15.83:0/1718018856 wait complete. 2023-11-28T11:09:15.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:15.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 Processor -- start 2023-11-28T11:09:15.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 -- start start 2023-11-28T11:09:15.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f06300aee20 0x7f06300b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06300af1f0 0x7f06300cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300b5b00 con 0x7f063006c370 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063006c800 con 0x7f06300af1f0 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.268+0000 7f0636fae700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063006cae0 con 0x7f06300aee20 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06300af1f0 0x7f06300cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06300af1f0 0x7f06300cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49616/0 (socket says 172.21.15.83:49616) 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 -- 172.21.15.83:0/1693300198 learned_addr learned my addr 172.21.15.83:0/1693300198 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 -- 172.21.15.83:0/1693300198 >> v2:172.21.15.90:3300/0 conn(0x7f06300aee20 msgr2=0x7f06300b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 --2- 172.21.15.83:0/1693300198 >> v2:172.21.15.90:3300/0 conn(0x7f06300aee20 0x7f06300b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 -- 172.21.15.83:0/1693300198 >> v1:172.21.15.90:6789/0 conn(0x7f063006c370 legacy=0x7f063006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 -- 172.21.15.83:0/1693300198 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063006cdc0 con 0x7f06300af1f0 2023-11-28T11:09:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f063554b700 1 --2- 172.21.15.83:0/1693300198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300af1f0 0x7f06300cc800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f06300715b0 tx=0x7f062800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305eb27470b1e817 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f062f7fe700 1 -- 172.21.15.83:0/1693300198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628010070 con 0x7f06300af1f0 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f062f7fe700 1 -- 172.21.15.83:0/1693300198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062800dd50 con 0x7f06300af1f0 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.269+0000 7f062f7fe700 1 -- 172.21.15.83:0/1693300198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800b310 con 0x7f06300af1f0 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 -- 172.21.15.83:0/1693300198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300af1f0 msgr2=0x7f06300cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 --2- 172.21.15.83:0/1693300198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300af1f0 0x7f06300cc800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f06300715b0 tx=0x7f062800d430 comp rx=0 tx=0).stop 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 -- 172.21.15.83:0/1693300198 shutdown_connections 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 --2- 172.21.15.83:0/1693300198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300af1f0 0x7f06300cc800 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 --2- 172.21.15.83:0/1693300198 >> v2:172.21.15.90:3300/0 conn(0x7f06300aee20 0x7f06300b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 -- 172.21.15.83:0/1693300198 >> 172.21.15.83:0/1693300198 conn(0x7f06300b2b60 msgr2=0x7f06300b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 -- 172.21.15.83:0/1693300198 shutdown_connections 2023-11-28T11:09:15.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.270+0000 7f0636fae700 1 -- 172.21.15.83:0/1693300198 wait complete. 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.271+0000 7f0636fae700 1 Processor -- start 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.271+0000 7f0636fae700 1 -- start start 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0636fae700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f063006c370 0x7f0630157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0636fae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 0x7f0630155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0636fae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06300af1f0 0x7f06300cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0636fae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06301556a0 con 0x7f06300aee20 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0636fae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063014c2d0 con 0x7f06300af1f0 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0636fae700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0630156690 con 0x7f063006c370 2023-11-28T11:09:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 0x7f0630155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 0x7f0630155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49618/0 (socket says 172.21.15.83:49618) 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 -- 172.21.15.83:0/748663755 learned_addr learned my addr 172.21.15.83:0/748663755 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f063554b700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06300af1f0 0x7f06300cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0634d4a700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f063006c370 0x7f0630157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 -- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f063006c370 msgr2=0x7f0630157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f063006c370 0x7f0630157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 -- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06300af1f0 msgr2=0x7f06300cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06300af1f0 0x7f06300cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 -- 172.21.15.83:0/748663755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062800bd40 con 0x7f06300aee20 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f0634d4a700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f063006c370 0x7f0630157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f063554b700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06300af1f0 0x7f06300cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062ffff700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 0x7f0630155160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c4a0 tx=0x7f061c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.272+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0054e0 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.273+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c003680 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.273+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063006fc70 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.273+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c003a10 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.273+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063006d0a0 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.274+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063006d380 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.274+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f061c010070 con 0x7f06300aee20 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.274+0000 7f062d7fa700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06140682f0 0x7f061406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.274+0000 7f0634d4a700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06140682f0 0x7f061406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.275+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f061c085050 con 0x7f06300aee20 2023-11-28T11:09:15.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.275+0000 7f0634d4a700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06140682f0 0x7f061406a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0624006fd0 tx=0x7f0624008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.279+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f061c051540 con 0x7f06300aee20 2023-11-28T11:09:15.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.459+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06300742b0 con 0x7f06300aee20 2023-11-28T11:09:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.460+0000 7f062d7fa700 1 -- 172.21.15.83:0/748663755 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f061c0170a0 con 0x7f06300aee20 2023-11-28T11:09:15.460 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:15.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.461+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06140682f0 msgr2=0x7f061406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.461+0000 7f0636fae700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06140682f0 0x7f061406a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0624006fd0 tx=0x7f0624008040 comp rx=0 tx=0).stop 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 msgr2=0x7f0630155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 0x7f0630155160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c4a0 tx=0x7f061c00c860 comp rx=0 tx=0).stop 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 shutdown_connections 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06300af1f0 0x7f06300cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f063006c370 0x7f0630157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06140682f0 0x7f061406a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 --2- 172.21.15.83:0/748663755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06300aee20 0x7f0630155160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 >> 172.21.15.83:0/748663755 conn(0x7f06300b2b60 msgr2=0x7f0630157f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 shutdown_connections 2023-11-28T11:09:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.462+0000 7f0636fae700 1 -- 172.21.15.83:0/748663755 wait complete. 2023-11-28T11:09:15.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.600+0000 7fbdfea55700 1 Processor -- start 2023-11-28T11:09:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.600+0000 7fbdfea55700 1 -- start start 2023-11-28T11:09:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdfea55700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf80ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdfea55700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf80cd940 con 0x7fbdf8158ad0 2023-11-28T11:09:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdfea55700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf80cdc20 con 0x7fbdf8071fa0 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdfea55700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80cd760 con 0x7fbdf80cd320 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdf7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbdf8158ad0 0x7fbdf8071890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47486/0 (socket says 172.21.15.83:47486) 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdf7fff700 1 -- 172.21.15.83:0/317929865 learned_addr learned my addr 172.21.15.83:0/317929865 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdfcff2700 1 --2- 172.21.15.83:0/317929865 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf80ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413565117 0 0) 0x7fbdf80cd940 con 0x7fbdf8158ad0 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde00035b0 con 0x7fbdf8158ad0 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.601+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270112434 0 0) 0x7fbdf80cdc20 con 0x7fbdf8071fa0 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf80cd940 con 0x7fbdf8071fa0 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdfcff2700 1 -- 172.21.15.83:0/317929865 >> v1:172.21.15.90:6789/0 conn(0x7fbdf8071fa0 legacy=0x7fbdf80ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdfcff2700 1 -- 172.21.15.83:0/317929865 >> v1:172.21.15.83:6789/0 conn(0x7fbdf8158ad0 legacy=0x7fbdf8071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdfcff2700 1 -- 172.21.15.83:0/317929865 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf80cdf00 con 0x7fbdf80cd320 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdfcff2700 1 --2- 172.21.15.83:0/317929865 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf80ce9e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000aed0 tx=0x7fbdf000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31cecec03994fb89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000f070 con 0x7fbdf80cd320 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf000a2e0 con 0x7fbdf80cd320 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdf6ffd700 1 -- 172.21.15.83:0/317929865 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000cbd0 con 0x7fbdf80cd320 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.602+0000 7fbdfea55700 1 -- 172.21.15.83:0/317929865 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 msgr2=0x7fbdf80ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.603+0000 7fbdfea55700 1 --2- 172.21.15.83:0/317929865 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf80ce9e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000aed0 tx=0x7fbdf000c330 comp rx=0 tx=0).stop 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.603+0000 7fbdfea55700 1 -- 172.21.15.83:0/317929865 shutdown_connections 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.603+0000 7fbdfea55700 1 --2- 172.21.15.83:0/317929865 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf80ce9e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.603+0000 7fbdfea55700 1 -- 172.21.15.83:0/317929865 >> 172.21.15.83:0/317929865 conn(0x7fbdf80cba50 msgr2=0x7fbdf80cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.603+0000 7fbdfea55700 1 -- 172.21.15.83:0/317929865 shutdown_connections 2023-11-28T11:09:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.603+0000 7fbdfea55700 1 -- 172.21.15.83:0/317929865 wait complete. 2023-11-28T11:09:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.604+0000 7fbdfea55700 1 Processor -- start 2023-11-28T11:09:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.604+0000 7fbdfea55700 1 -- start start 2023-11-28T11:09:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8071fa0 0x7fbdf8158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf81e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 0x7fbdf81e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf81e4690 con 0x7fbdf8158ad0 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf809f3f0 con 0x7fbdf8071fa0 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf809f6d0 con 0x7fbdf80cd320 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 0x7fbdf81e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 0x7fbdf81e51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49636/0 (socket says 172.21.15.83:49636) 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 -- 172.21.15.83:0/2696980110 learned_addr learned my addr 172.21.15.83:0/2696980110 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdf7fff700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8071fa0 0x7fbdf8158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdf77fe700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf81e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 -- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 msgr2=0x7fbdf81e4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf81e4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 -- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8071fa0 msgr2=0x7fbdf8158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8071fa0 0x7fbdf8158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 -- 172.21.15.83:0/2696980110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf000ab80 con 0x7fbdf8158ad0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdf7fff700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8071fa0 0x7fbdf8158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfcff2700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 0x7fbdf81e51f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80673a0 tx=0x7fbdf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0004450 con 0x7fbdf8158ad0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0003e90 con 0x7fbdf8158ad0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf80ce4c0 con 0x7fbdf8158ad0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00047f0 con 0x7fbdf8158ad0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.605+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf80ce1e0 con 0x7fbdf8158ad0 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.606+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8158ea0 con 0x7fbdf8158ad0 2023-11-28T11:09:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.607+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbdf0024030 con 0x7fbdf8158ad0 2023-11-28T11:09:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.607+0000 7fbdf4ff9700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbde40682f0 0x7fbde406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.607+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbdf0087020 con 0x7fbdf8158ad0 2023-11-28T11:09:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.607+0000 7fbdf7fff700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbde40682f0 0x7fbde406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.607+0000 7fbdf7fff700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbde40682f0 0x7fbde406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f60 tx=0x7fbdec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.611+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbdf0053510 con 0x7fbdf8158ad0 2023-11-28T11:09:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.792+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbdf80742b0 con 0x7fbdf8158ad0 2023-11-28T11:09:15.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.792+0000 7fbdf4ff9700 1 -- 172.21.15.83:0/2696980110 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbdf0056b30 con 0x7fbdf8158ad0 2023-11-28T11:09:15.793 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:15.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbde40682f0 msgr2=0x7fbde406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbde40682f0 0x7fbde406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f60 tx=0x7fbdec009040 comp rx=0 tx=0).stop 2023-11-28T11:09:15.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 msgr2=0x7fbdf81e51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 0x7fbdf81e51f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80673a0 tx=0x7fbdf0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 shutdown_connections 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbde40682f0 0x7fbde406a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf8158ad0 0x7fbdf81e51f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8071fa0 0x7fbdf8158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 --2- 172.21.15.83:0/2696980110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdf80cd320 0x7fbdf81e4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.794+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 >> 172.21.15.83:0/2696980110 conn(0x7fbdf80cba50 msgr2=0x7fbdf8159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.795+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 shutdown_connections 2023-11-28T11:09:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.795+0000 7fbdfea55700 1 -- 172.21.15.83:0/2696980110 wait complete. 2023-11-28T11:09:15.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:15.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.928+0000 7f3bf44ed700 1 Processor -- start 2023-11-28T11:09:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.928+0000 7f3bf44ed700 1 -- start start 2023-11-28T11:09:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf44ed700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bec15d0e0 0x7f3bec0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf44ed700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0cbfb0 con 0x7f3bec15cd10 2023-11-28T11:09:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf44ed700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0cb710 con 0x7f3bec0b1fb0 2023-11-28T11:09:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf44ed700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0cb9f0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf1a88700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bec15d0e0 0x7f3bec0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf2289700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3bec15cd10 0x7f3bec0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47500/0 (socket says 172.21.15.83:47500) 2023-11-28T11:09:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.929+0000 7f3bf2289700 1 -- 172.21.15.83:0/2338213626 learned_addr learned my addr 172.21.15.83:0/2338213626 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1a88700 1 -- 172.21.15.83:0/2338213626 >> v1:172.21.15.90:6789/0 conn(0x7f3bec0b1fb0 legacy=0x7f3bec06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1a88700 1 -- 172.21.15.83:0/2338213626 >> v1:172.21.15.83:6789/0 conn(0x7f3bec15cd10 legacy=0x7f3bec0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1287700 1 -- 172.21.15.83:0/2338213626 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268028949 0 0) 0x7f3bec0cbfb0 con 0x7f3bec15cd10 2023-11-28T11:09:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1a88700 1 -- 172.21.15.83:0/2338213626 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0cbcd0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1a88700 1 --2- 172.21.15.83:0/2338213626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0ca970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009840 tx=0x7f3be8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c81615032331422 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1287700 1 -- 172.21.15.83:0/2338213626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8011830 con 0x7f3bec15d0e0 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1287700 1 -- 172.21.15.83:0/2338213626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8011990 con 0x7f3bec15d0e0 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf1287700 1 -- 172.21.15.83:0/2338213626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8011d20 con 0x7f3bec15d0e0 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf44ed700 1 -- 172.21.15.83:0/2338213626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 msgr2=0x7f3bec0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/2338213626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0ca970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009840 tx=0x7f3be8009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf44ed700 1 -- 172.21.15.83:0/2338213626 shutdown_connections 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/2338213626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0ca970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.930+0000 7f3bf44ed700 1 -- 172.21.15.83:0/2338213626 >> 172.21.15.83:0/2338213626 conn(0x7f3bec06e9a0 msgr2=0x7f3bec0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.931+0000 7f3bf44ed700 1 -- 172.21.15.83:0/2338213626 shutdown_connections 2023-11-28T11:09:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.931+0000 7f3bf44ed700 1 -- 172.21.15.83:0/2338213626 wait complete. 2023-11-28T11:09:15.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 Processor -- start 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 -- start start 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bec0b1fb0 0x7f3bec0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec15cd10 0x7f3bec1f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b5790 con 0x7f3bec15d0e0 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b41b0 con 0x7f3bec15cd10 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.932+0000 7f3bf44ed700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b4490 con 0x7f3bec0b1fb0 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49650/0 (socket says 172.21.15.83:49650) 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 -- 172.21.15.83:0/3458101949 learned_addr learned my addr 172.21.15.83:0/3458101949 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf1a88700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec15cd10 0x7f3bec1f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2289700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bec0b1fb0 0x7f3bec0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 -- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bec0b1fb0 msgr2=0x7f3bec0b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bec0b1fb0 0x7f3bec0b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 -- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec15cd10 msgr2=0x7f3bec1f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec15cd10 0x7f3bec1f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 -- 172.21.15.83:0/3458101949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80094a0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf1a88700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec15cd10 0x7f3bec1f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf2a8a700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0c63a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3be400cbc0 tx=0x7f3be400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4003c00 con 0x7f3bec15d0e0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4003d60 con 0x7f3bec15d0e0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec06b660 con 0x7f3bec15d0e0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400abf0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.933+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec06b9b0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.934+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec0cd9f0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.934+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3be4010050 con 0x7f3bec15d0e0 2023-11-28T11:09:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.935+0000 7f3be2ffd700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3bd00682f0 0x7f3bd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.935+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3be4085ea0 con 0x7f3bec15d0e0 2023-11-28T11:09:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.935+0000 7f3bf2289700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3bd00682f0 0x7f3bd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.935+0000 7f3bf2289700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3bd00682f0 0x7f3bd006a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc007fd0 tx=0x7f3bdc007c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:15.939+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3be4052410 con 0x7f3bec15d0e0 2023-11-28T11:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.123+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bec074320 con 0x7f3bec15d0e0 2023-11-28T11:09:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.124+0000 7f3be2ffd700 1 -- 172.21.15.83:0/3458101949 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f3be4055a30 con 0x7f3bec15d0e0 2023-11-28T11:09:16.125 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:16.125 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3bd00682f0 msgr2=0x7f3bd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3bd00682f0 0x7f3bd006a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc007fd0 tx=0x7f3bdc007c20 comp rx=0 tx=0).stop 2023-11-28T11:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 msgr2=0x7f3bec0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0c63a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3be400cbc0 tx=0x7f3be400cf80 comp rx=0 tx=0).stop 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 shutdown_connections 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3bd00682f0 0x7f3bd006a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec15d0e0 0x7f3bec0c63a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bec0b1fb0 0x7f3bec0b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 --2- 172.21.15.83:0/3458101949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec15cd10 0x7f3bec1f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 >> 172.21.15.83:0/3458101949 conn(0x7f3bec06e9a0 msgr2=0x7f3bec0cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 shutdown_connections 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.127+0000 7f3bf44ed700 1 -- 172.21.15.83:0/3458101949 wait complete. 2023-11-28T11:09:16.140 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:16.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:16.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.263+0000 7f5b98e79700 1 Processor -- start 2023-11-28T11:09:16.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.263+0000 7f5b98e79700 1 -- start start 2023-11-28T11:09:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b98e79700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b94158040 0x7f5b9415b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b98e79700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5b940d0560 0x7f5b940ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b98e79700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b940cd4f0 0x7f5b940b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b98e79700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940cea60 con 0x7f5b940cd4f0 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b98e79700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940b3660 con 0x7f5b940d0560 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b98e79700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9415ba90 con 0x7f5b94158040 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.264+0000 7f5b92d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b940cd4f0 0x7f5b940b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b940cd4f0 0x7f5b940b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49658/0 (socket says 172.21.15.83:49658) 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 -- 172.21.15.83:0/3729951673 learned_addr learned my addr 172.21.15.83:0/3729951673 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b91d9b700 1 --2- 172.21.15.83:0/3729951673 >> v2:172.21.15.90:3300/0 conn(0x7f5b940d0560 0x7f5b940ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b9259c700 1 --2- 172.21.15.83:0/3729951673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b94158040 0x7f5b9415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 -- 172.21.15.83:0/3729951673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b94158040 msgr2=0x7f5b9415b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 --2- 172.21.15.83:0/3729951673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b94158040 0x7f5b9415b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 -- 172.21.15.83:0/3729951673 >> v2:172.21.15.90:3300/0 conn(0x7f5b940d0560 msgr2=0x7f5b940ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 --2- 172.21.15.83:0/3729951673 >> v2:172.21.15.90:3300/0 conn(0x7f5b940d0560 0x7f5b940ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 -- 172.21.15.83:0/3729951673 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940d0a00 con 0x7f5b940cd4f0 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b91d9b700 1 --2- 172.21.15.83:0/3729951673 >> v2:172.21.15.90:3300/0 conn(0x7f5b940d0560 0x7f5b940ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b92d9d700 1 --2- 172.21.15.83:0/3729951673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940b27e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b9406e970 tx=0x7f5b8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bb6d7f2f385dfa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b9159a700 1 -- 172.21.15.83:0/3729951673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00e070 con 0x7f5b940cd4f0 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b9159a700 1 -- 172.21.15.83:0/3729951673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c00b9c0 con 0x7f5b940cd4f0 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.265+0000 7f5b9159a700 1 -- 172.21.15.83:0/3729951673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00bd70 con 0x7f5b940cd4f0 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 -- 172.21.15.83:0/3729951673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 msgr2=0x7f5b940b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 --2- 172.21.15.83:0/3729951673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940b27e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b9406e970 tx=0x7f5b8c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 -- 172.21.15.83:0/3729951673 shutdown_connections 2023-11-28T11:09:16.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 --2- 172.21.15.83:0/3729951673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b94158040 0x7f5b9415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 --2- 172.21.15.83:0/3729951673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940b27e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 --2- 172.21.15.83:0/3729951673 >> v2:172.21.15.90:3300/0 conn(0x7f5b940d0560 0x7f5b940ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 -- 172.21.15.83:0/3729951673 >> 172.21.15.83:0/3729951673 conn(0x7f5b9406de20 msgr2=0x7f5b9406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 -- 172.21.15.83:0/3729951673 shutdown_connections 2023-11-28T11:09:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.266+0000 7f5b98e79700 1 -- 172.21.15.83:0/3729951673 wait complete. 2023-11-28T11:09:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.267+0000 7f5b98e79700 1 Processor -- start 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.267+0000 7f5b98e79700 1 -- start start 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b98e79700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b98e79700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b940d0560 0x7f5b940ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b98e79700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b94158040 0x7f5b940b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b98e79700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940c4400 con 0x7f5b940cd4f0 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b98e79700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940b47a0 con 0x7f5b94158040 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b98e79700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940c7200 con 0x7f5b940d0560 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49660/0 (socket says 172.21.15.83:49660) 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 -- 172.21.15.83:0/2106322780 learned_addr learned my addr 172.21.15.83:0/2106322780 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b92d9d700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b94158040 0x7f5b940b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b91d9b700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b940d0560 0x7f5b940ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 -- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b940d0560 msgr2=0x7f5b940ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b940d0560 0x7f5b940ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 -- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b94158040 msgr2=0x7f5b940b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b94158040 0x7f5b940b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 -- 172.21.15.83:0/2106322780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c009b30 con 0x7f5b940cd4f0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.268+0000 7f5b9259c700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940c62d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400d800 tx=0x7f5b8400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.269+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840125f0 con 0x7f5b940cd4f0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.269+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84012750 con 0x7f5b940cd4f0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.269+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b940cb730 con 0x7f5b940cd4f0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.269+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84012b00 con 0x7f5b940cd4f0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.269+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b940cb450 con 0x7f5b940cd4f0 2023-11-28T11:09:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.269+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94158410 con 0x7f5b940cd4f0 2023-11-28T11:09:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.270+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b84012c60 con 0x7f5b940cd4f0 2023-11-28T11:09:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.270+0000 7f5b8affd700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b740682f0 0x7f5b7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.270+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b84086090 con 0x7f5b940cd4f0 2023-11-28T11:09:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.270+0000 7f5b91d9b700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b740682f0 0x7f5b7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.271+0000 7f5b91d9b700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b740682f0 0x7f5b7406a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b940cefb0 tx=0x7f5b7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.275+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b84052600 con 0x7f5b940cd4f0 2023-11-28T11:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.455+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b940742b0 con 0x7f5b940cd4f0 2023-11-28T11:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.455+0000 7f5b8affd700 1 -- 172.21.15.83:0/2106322780 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5b84055c20 con 0x7f5b940cd4f0 2023-11-28T11:09:16.455 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b740682f0 msgr2=0x7f5b7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b740682f0 0x7f5b7406a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b940cefb0 tx=0x7f5b7c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 msgr2=0x7f5b940c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940c62d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400d800 tx=0x7f5b8400db10 comp rx=0 tx=0).stop 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 shutdown_connections 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b94158040 0x7f5b940b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5b740682f0 0x7f5b7406a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b940cd4f0 0x7f5b940c62d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 --2- 172.21.15.83:0/2106322780 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b940d0560 0x7f5b940ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 >> 172.21.15.83:0/2106322780 conn(0x7f5b9406de20 msgr2=0x7f5b940cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 shutdown_connections 2023-11-28T11:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.457+0000 7f5b98e79700 1 -- 172.21.15.83:0/2106322780 wait complete. 2023-11-28T11:09:16.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:16.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.592+0000 7f0e8e146700 1 Processor -- start 2023-11-28T11:09:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.592+0000 7f0e8e146700 1 -- start start 2023-11-28T11:09:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e8e146700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e8e146700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0e880b4bb0 0x7f0e8806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e8e146700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e8806e1a0 0x7f0e881604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e8e146700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8806e620 con 0x7f0e8806e1a0 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e8e146700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880b5020 con 0x7f0e880b4bb0 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e8e146700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8806d4a0 con 0x7f0e8815c890 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e8806e1a0 0x7f0e881604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e8806e1a0 0x7f0e881604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49672/0 (socket says 172.21.15.83:49672) 2023-11-28T11:09:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 -- 172.21.15.83:0/2284029137 learned_addr learned my addr 172.21.15.83:0/2284029137 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 -- 172.21.15.83:0/2284029137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 msgr2=0x7f0e880aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e877fe700 1 --2- 172.21.15.83:0/2284029137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e86ffd700 1 --2- 172.21.15.83:0/2284029137 >> v2:172.21.15.90:3300/0 conn(0x7f0e880b4bb0 0x7f0e8806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 --2- 172.21.15.83:0/2284029137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 -- 172.21.15.83:0/2284029137 >> v2:172.21.15.90:3300/0 conn(0x7f0e880b4bb0 msgr2=0x7f0e8806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 --2- 172.21.15.83:0/2284029137 >> v2:172.21.15.90:3300/0 conn(0x7f0e880b4bb0 0x7f0e8806dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 -- 172.21.15.83:0/2284029137 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880b5ae0 con 0x7f0e8806e1a0 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.593+0000 7f0e87fff700 1 --2- 172.21.15.83:0/2284029137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e8806e1a0 0x7f0e881604e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009e50 tx=0x7f0e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aae0416c275fbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e867fc700 1 -- 172.21.15.83:0/2284029137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000e070 con 0x7f0e8806e1a0 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e867fc700 1 -- 172.21.15.83:0/2284029137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000b980 con 0x7f0e8806e1a0 2023-11-28T11:09:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e867fc700 1 -- 172.21.15.83:0/2284029137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000bd30 con 0x7f0e8806e1a0 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 -- 172.21.15.83:0/2284029137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e8806e1a0 msgr2=0x7f0e881604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 --2- 172.21.15.83:0/2284029137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e8806e1a0 0x7f0e881604e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009e50 tx=0x7f0e8000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 -- 172.21.15.83:0/2284029137 shutdown_connections 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 --2- 172.21.15.83:0/2284029137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 --2- 172.21.15.83:0/2284029137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e8806e1a0 0x7f0e881604e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 --2- 172.21.15.83:0/2284029137 >> v2:172.21.15.90:3300/0 conn(0x7f0e880b4bb0 0x7f0e8806dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 -- 172.21.15.83:0/2284029137 >> 172.21.15.83:0/2284029137 conn(0x7f0e880b2b60 msgr2=0x7f0e880b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 -- 172.21.15.83:0/2284029137 shutdown_connections 2023-11-28T11:09:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.594+0000 7f0e8e146700 1 -- 172.21.15.83:0/2284029137 wait complete. 2023-11-28T11:09:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.595+0000 7f0e8e146700 1 Processor -- start 2023-11-28T11:09:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 -- start start 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e8806e1a0 0x7f0e88155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 0x7f0e880cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88156090 con 0x7f0e880b4bb0 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880cc2e0 con 0x7f0e8806e1a0 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e8e146700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880cc5c0 con 0x7f0e8815c890 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 0x7f0e880cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.596+0000 7f0e86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 0x7f0e880cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49674/0 (socket says 172.21.15.83:49674) 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 -- 172.21.15.83:0/1356455597 learned_addr learned my addr 172.21.15.83:0/1356455597 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e87fff700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e877fe700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e8806e1a0 0x7f0e88155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 -- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 msgr2=0x7f0e880c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 -- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e8806e1a0 msgr2=0x7f0e88155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e8806e1a0 0x7f0e88155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 -- 172.21.15.83:0/1356455597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80009b30 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e86ffd700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 0x7f0e880cbc50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c7f0 tx=0x7f0e7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78011800 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78011960 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e880b5240 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78011d10 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.597+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8806c400 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.598+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e880b5520 con 0x7f0e880b4bb0 2023-11-28T11:09:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.598+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e78017030 con 0x7f0e880b4bb0 2023-11-28T11:09:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.599+0000 7f0e6ffff700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0e740682f0 0x7f0e7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.599+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0e78085990 con 0x7f0e880b4bb0 2023-11-28T11:09:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.599+0000 7f0e877fe700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0e740682f0 0x7f0e7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.599+0000 7f0e877fe700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0e740682f0 0x7f0e7406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00b840 tx=0x7f0e7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.603+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e78051f00 con 0x7f0e880b4bb0 2023-11-28T11:09:16.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.783+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e880742b0 con 0x7f0e880b4bb0 2023-11-28T11:09:16.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.783+0000 7f0e6ffff700 1 -- 172.21.15.83:0/1356455597 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e78055520 con 0x7f0e880b4bb0 2023-11-28T11:09:16.784 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.785+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0e740682f0 msgr2=0x7f0e7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.785+0000 7f0e8e146700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0e740682f0 0x7f0e7406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00b840 tx=0x7f0e7c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.785+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 msgr2=0x7f0e880cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.785+0000 7f0e8e146700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 0x7f0e880cbc50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c7f0 tx=0x7f0e7800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 shutdown_connections 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e8815c890 0x7f0e880c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e8806e1a0 0x7f0e88155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0e740682f0 0x7f0e7406a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 --2- 172.21.15.83:0/1356455597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e880b4bb0 0x7f0e880cbc50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 >> 172.21.15.83:0/1356455597 conn(0x7f0e880b2b60 msgr2=0x7f0e880ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 shutdown_connections 2023-11-28T11:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.786+0000 7f0e8e146700 1 -- 172.21.15.83:0/1356455597 wait complete. 2023-11-28T11:09:16.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:16.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.921+0000 7fd9a7e3c700 1 Processor -- start 2023-11-28T11:09:16.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.921+0000 7fd9a7e3c700 1 -- start start 2023-11-28T11:09:16.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a7e3c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd9a015c890 0x7fd9a00aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a7e3c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9a00b4bb0 0x7fd9a006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a7e3c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a006e1a0 0x7fd9a01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a7e3c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a006e620 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a7e3c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a00b5020 con 0x7fd9a015c890 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a7e3c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a006d4a0 con 0x7fd9a006e1a0 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a53d7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9a00b4bb0 0x7fd9a006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a53d7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9a00b4bb0 0x7fd9a006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49682/0 (socket says 172.21.15.83:49682) 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a53d7700 1 -- 172.21.15.83:0/972824843 learned_addr learned my addr 172.21.15.83:0/972824843 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a5bd8700 1 --2- 172.21.15.83:0/972824843 >> v2:172.21.15.90:3300/0 conn(0x7fd9a015c890 0x7fd9a00aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.922+0000 7fd9a63d9700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a006e1a0 0x7fd9a01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a53d7700 1 -- 172.21.15.83:0/972824843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a006e1a0 msgr2=0x7fd9a01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a53d7700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a006e1a0 0x7fd9a01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a53d7700 1 -- 172.21.15.83:0/972824843 >> v2:172.21.15.90:3300/0 conn(0x7fd9a015c890 msgr2=0x7fd9a00aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a53d7700 1 --2- 172.21.15.83:0/972824843 >> v2:172.21.15.90:3300/0 conn(0x7fd9a015c890 0x7fd9a00aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a53d7700 1 -- 172.21.15.83:0/972824843 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a00b5ae0 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a63d9700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a006e1a0 0x7fd9a01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a53d7700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a006dc60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00715b0 tx=0x7fd99c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb22327af7bb4be1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a4bd6700 1 -- 172.21.15.83:0/972824843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011830 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a4bd6700 1 -- 172.21.15.83:0/972824843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c011990 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a4bd6700 1 -- 172.21.15.83:0/972824843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011d20 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/972824843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 msgr2=0x7fd9a006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a006dc60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00715b0 tx=0x7fd99c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.923+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/972824843 shutdown_connections 2023-11-28T11:09:16.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.924+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a006e1a0 0x7fd9a01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.924+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/972824843 >> v2:172.21.15.90:3300/0 conn(0x7fd9a015c890 0x7fd9a00aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.924+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/972824843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a006dc60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.924+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/972824843 >> 172.21.15.83:0/972824843 conn(0x7fd9a00b2b60 msgr2=0x7fd9a00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.924+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/972824843 shutdown_connections 2023-11-28T11:09:16.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.924+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/972824843 wait complete. 2023-11-28T11:09:16.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.925+0000 7fd9a7e3c700 1 Processor -- start 2023-11-28T11:09:16.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.925+0000 7fd9a7e3c700 1 -- start start 2023-11-28T11:09:16.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.925+0000 7fd9a7e3c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9a006e1a0 0x7fd9a00c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a00cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a015c890 0x7fd9a00c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00d0170 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00cf630 con 0x7fd9a006e1a0 2023-11-28T11:09:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00cf8d0 con 0x7fd9a015c890 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a00cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a00cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49690/0 (socket says 172.21.15.83:49690) 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 -- 172.21.15.83:0/1337214597 learned_addr learned my addr 172.21.15.83:0/1337214597 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a63d9700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a015c890 0x7fd9a00c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a5bd8700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9a006e1a0 0x7fd9a00c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 -- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a015c890 msgr2=0x7fd9a00c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a015c890 0x7fd9a00c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 -- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9a006e1a0 msgr2=0x7fd9a00c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9a006e1a0 0x7fd9a00c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 -- 172.21.15.83:0/1337214597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009550 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a63d9700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a015c890 0x7fd9a00c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a53d7700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a00cf0f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd99c014040 tx=0x7fd99c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004510 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c01e070 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a00b5240 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0038a0 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.926+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a006c400 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.927+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a00b5520 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.928+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd99c00f050 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.928+0000 7fd9967fc700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd984068320 0x7fd98406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.928+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd99c020070 con 0x7fd9a00b4bb0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.928+0000 7fd9a5bd8700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd984068320 0x7fd98406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.928+0000 7fd9a5bd8700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd984068320 0x7fd98406a7d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd990009b20 tx=0x7fd9900091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:16.932+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd99c0536e0 con 0x7fd9a00b4bb0 2023-11-28T11:09:17.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.118+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9a00cd160 con 0x7fd9a00b4bb0 2023-11-28T11:09:17.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.119+0000 7fd9967fc700 1 -- 172.21.15.83:0/1337214597 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd99c015020 con 0x7fd9a00b4bb0 2023-11-28T11:09:17.119 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.120+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd984068320 msgr2=0x7fd98406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.120+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd984068320 0x7fd98406a7d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd990009b20 tx=0x7fd9900091c0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.120+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 msgr2=0x7fd9a00cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.120+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a00cf0f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd99c014040 tx=0x7fd99c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 shutdown_connections 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9a015c890 0x7fd9a00c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9a006e1a0 0x7fd9a00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd984068320 0x7fd98406a7d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 --2- 172.21.15.83:0/1337214597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a00b4bb0 0x7fd9a00cf0f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 >> 172.21.15.83:0/1337214597 conn(0x7fd9a00b2b60 msgr2=0x7fd9a006c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 shutdown_connections 2023-11-28T11:09:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.121+0000 7fd9a7e3c700 1 -- 172.21.15.83:0/1337214597 wait complete. 2023-11-28T11:09:17.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.254+0000 7f672af3b700 1 Processor -- start 2023-11-28T11:09:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.254+0000 7f672af3b700 1 -- start start 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f672af3b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6724159a20 0x7f67240cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f672af3b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67240b5680 0x7f672406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f672af3b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67240cb3a0 con 0x7f67240b1690 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f672af3b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240b54f0 con 0x7f67240b5680 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f672af3b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240cb640 con 0x7f6724159a20 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67294d8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67240b5680 0x7f672406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f6728cd7700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f67240b1690 0x7f67240b1a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47556/0 (socket says 172.21.15.83:47556) 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f6728cd7700 1 -- 172.21.15.83:0/1574671707 learned_addr learned my addr 172.21.15.83:0/1574671707 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f6723fff700 1 --2- 172.21.15.83:0/1574671707 >> v2:172.21.15.90:3300/0 conn(0x7f6724159a20 0x7f67240cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67294d8700 1 -- 172.21.15.83:0/1574671707 >> v2:172.21.15.90:3300/0 conn(0x7f6724159a20 msgr2=0x7f67240cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67294d8700 1 --2- 172.21.15.83:0/1574671707 >> v2:172.21.15.90:3300/0 conn(0x7f6724159a20 0x7f67240cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67294d8700 1 -- 172.21.15.83:0/1574671707 >> v1:172.21.15.83:6789/0 conn(0x7f67240b1690 legacy=0x7f67240b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67294d8700 1 -- 172.21.15.83:0/1574671707 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240cb920 con 0x7f67240b5680 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67237fe700 1 -- 172.21.15.83:0/1574671707 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402307596 0 0) 0x7f67240cb3a0 con 0x7f67240b1690 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67294d8700 1 --2- 172.21.15.83:0/1574671707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b5680 0x7f672406bca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67240b4ed0 tx=0x7f671c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee47eae7ae09d57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.255+0000 7f67237fe700 1 -- 172.21.15.83:0/1574671707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00e070 con 0x7f67240b5680 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f67237fe700 1 -- 172.21.15.83:0/1574671707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c00bbd0 con 0x7f67240b5680 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f67237fe700 1 -- 172.21.15.83:0/1574671707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c012430 con 0x7f67240b5680 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 -- 172.21.15.83:0/1574671707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b5680 msgr2=0x7f672406bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 --2- 172.21.15.83:0/1574671707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b5680 0x7f672406bca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67240b4ed0 tx=0x7f671c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 -- 172.21.15.83:0/1574671707 shutdown_connections 2023-11-28T11:09:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 --2- 172.21.15.83:0/1574671707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b5680 0x7f672406bca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 --2- 172.21.15.83:0/1574671707 >> v2:172.21.15.90:3300/0 conn(0x7f6724159a20 0x7f67240cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 -- 172.21.15.83:0/1574671707 >> 172.21.15.83:0/1574671707 conn(0x7f67240cc690 msgr2=0x7f67240cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 -- 172.21.15.83:0/1574671707 shutdown_connections 2023-11-28T11:09:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.256+0000 7f672af3b700 1 -- 172.21.15.83:0/1574671707 wait complete. 2023-11-28T11:09:17.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.257+0000 7f672af3b700 1 Processor -- start 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.257+0000 7f672af3b700 1 -- start start 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f672af3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 0x7f672414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f672af3b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f67240b5680 0x7f67240c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f672af3b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6724159a20 0x7f67240c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f672af3b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67241511a0 con 0x7f67240b1690 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f672af3b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6724152400 con 0x7f6724159a20 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f672af3b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67241517f0 con 0x7f67240b5680 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f6728cd7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 0x7f672414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f6728cd7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 0x7f672414f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49702/0 (socket says 172.21.15.83:49702) 2023-11-28T11:09:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f6728cd7700 1 -- 172.21.15.83:0/17023784 learned_addr learned my addr 172.21.15.83:0/17023784 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f6723fff700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f67240b5680 0x7f67240c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.258+0000 7f67294d8700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6724159a20 0x7f67240c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6728cd7700 1 -- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f67240b5680 msgr2=0x7f67240c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6728cd7700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f67240b5680 0x7f67240c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6728cd7700 1 -- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6724159a20 msgr2=0x7f67240c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6728cd7700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6724159a20 0x7f67240c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6728cd7700 1 -- 172.21.15.83:0/17023784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009a70 con 0x7f67240b1690 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6723fff700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f67240b5680 0x7f67240c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f67294d8700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6724159a20 0x7f67240c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f6728cd7700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 0x7f672414f160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67180011e0 tx=0x7f671800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718004c80 con 0x7f67240b1690 2023-11-28T11:09:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6718004e00 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67240cbee0 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718007310 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.259+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67240cbc00 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.260+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67240cc1c0 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.260+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f671801a020 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.260+0000 7f67217fa700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f67080682f0 0x7f670806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.261+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6718085bd0 con 0x7f67240b1690 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.261+0000 7f6723fff700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f67080682f0 0x7f670806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.261+0000 7f6723fff700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f67080682f0 0x7f670806a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67241e3d30 tx=0x7f6710008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.265+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6718052140 con 0x7f67240b1690 2023-11-28T11:09:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.448+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67240742b0 con 0x7f67240b1690 2023-11-28T11:09:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.448+0000 7f67217fa700 1 -- 172.21.15.83:0/17023784 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6718055760 con 0x7f67240b1690 2023-11-28T11:09:17.449 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f67080682f0 msgr2=0x7f670806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f67080682f0 0x7f670806a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67241e3d30 tx=0x7f6710008040 comp rx=0 tx=0).stop 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 msgr2=0x7f672414f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 0x7f672414f160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67180011e0 tx=0x7f671800cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 shutdown_connections 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6724159a20 0x7f67240c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f67080682f0 0x7f670806a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67240b1690 0x7f672414f160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 --2- 172.21.15.83:0/17023784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f67240b5680 0x7f67240c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 >> 172.21.15.83:0/17023784 conn(0x7f67240cc690 msgr2=0x7f6724158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 shutdown_connections 2023-11-28T11:09:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.450+0000 7f672af3b700 1 -- 172.21.15.83:0/17023784 wait complete. 2023-11-28T11:09:17.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.586+0000 7facc7b54700 1 Processor -- start 2023-11-28T11:09:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.586+0000 7facc7b54700 1 -- start start 2023-11-28T11:09:17.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc7b54700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 0x7facc00aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc7b54700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7facc00b4bb0 0x7facc006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc7b54700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc006d4a0 con 0x7facc015c890 2023-11-28T11:09:17.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc7b54700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc00b5050 con 0x7facc00b4bb0 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc7b54700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b5ae0 con 0x7facc006e370 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc58f0700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7facc015c890 0x7facc006dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47570/0 (socket says 172.21.15.83:47570) 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc58f0700 1 -- 172.21.15.83:0/4142454797 learned_addr learned my addr 172.21.15.83:0/4142454797 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615551743 0 0) 0x7facc006d4a0 con 0x7facc015c890 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc50ef700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 0x7facc00aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc60f1700 1 --2- 172.21.15.83:0/4142454797 >> v2:172.21.15.90:3300/0 conn(0x7facc00b4bb0 0x7facc006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facac0035b0 con 0x7facc015c890 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703532096 0 0) 0x7facac0035b0 con 0x7facc015c890 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 msgr2=0x7facc00aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 0x7facc00aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 >> v2:172.21.15.90:3300/0 conn(0x7facc00b4bb0 msgr2=0x7facc006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 --2- 172.21.15.83:0/4142454797 >> v2:172.21.15.90:3300/0 conn(0x7facc00b4bb0 0x7facc006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc006c400 con 0x7facc015c890 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.587+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7facbc0017f0 con 0x7facc015c890 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 >> v1:172.21.15.83:6789/0 conn(0x7facc015c890 legacy=0x7facc006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facac003a00 0x7facac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 0x7facac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac004d90 0x7facac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b5ae0 con 0x7facac004d90 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac00ba10 con 0x7facac003a00 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac00bb50 con 0x7facc006e370 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac004d90 0x7facac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc58f0700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facac003a00 0x7facac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc50ef700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 0x7facac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 -- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 msgr2=0x7facac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facc006e370 0x7facac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 -- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facac003a00 msgr2=0x7facac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facac003a00 0x7facac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 -- 172.21.15.83:0/4142454797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0035b0 con 0x7facac004d90 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc60f1700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac004d90 0x7facac005200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fc0 tx=0x7facac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f24aadac40b2b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800ec30 con 0x7facac004d90 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800f040 con 0x7facac004d90 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.588+0000 7facc48ee700 1 -- 172.21.15.83:0/4142454797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8014480 con 0x7facac004d90 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc7b54700 1 -- 172.21.15.83:0/4142454797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac004d90 msgr2=0x7facac005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc7b54700 1 --2- 172.21.15.83:0/4142454797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac004d90 0x7facac005200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fc0 tx=0x7facac005880 comp rx=0 tx=0).stop 2023-11-28T11:09:17.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc60f1700 1 -- 172.21.15.83:0/4142454797 reap_dead start 2023-11-28T11:09:17.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc7b54700 1 -- 172.21.15.83:0/4142454797 shutdown_connections 2023-11-28T11:09:17.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc7b54700 1 -- 172.21.15.83:0/4142454797 >> 172.21.15.83:0/4142454797 conn(0x7facc00b2b60 msgr2=0x7facc00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc7b54700 1 -- 172.21.15.83:0/4142454797 shutdown_connections 2023-11-28T11:09:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.589+0000 7facc7b54700 1 -- 172.21.15.83:0/4142454797 wait complete. 2023-11-28T11:09:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.590+0000 7facc7b54700 1 Processor -- start 2023-11-28T11:09:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.590+0000 7facc7b54700 1 -- start start 2023-11-28T11:09:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc7b54700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facac004d90 0x7facc014a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc7b54700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 0x7facc00c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc7b54700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facc00c7220 0x7facc00c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc7b54700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc014a620 con 0x7facc015a1f0 2023-11-28T11:09:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc7b54700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00c76a0 con 0x7facc00c7220 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc7b54700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00c7980 con 0x7facac004d90 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 0x7facc00c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 0x7facc00c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49724/0 (socket says 172.21.15.83:49724) 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 -- 172.21.15.83:0/2815285992 learned_addr learned my addr 172.21.15.83:0/2815285992 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc60f1700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facc00c7220 0x7facc00c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc58f0700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facac004d90 0x7facc014a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 -- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facac004d90 msgr2=0x7facc014a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facac004d90 0x7facc014a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 -- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facc00c7220 msgr2=0x7facc00c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facc00c7220 0x7facc00c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 -- 172.21.15.83:0/2815285992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00b5520 con 0x7facc015a1f0 2023-11-28T11:09:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc60f1700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facc00c7220 0x7facc00c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc58f0700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facac004d90 0x7facc014a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facc50ef700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 0x7facc00c6ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7facb00119c0 tx=0x7facb0011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.591+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0004240 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.592+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb00043a0 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.592+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00b5240 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.592+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000f730 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.592+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc00b5800 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.593+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc00aef40 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.593+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7facb0025070 con 0x7facc015a1f0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.593+0000 7facb67fc700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faca40682f0 0x7faca406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.594+0000 7facc58f0700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faca40682f0 0x7faca406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.594+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7facb0086440 con 0x7facc015a1f0 2023-11-28T11:09:17.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.594+0000 7facc58f0700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faca40682f0 0x7faca406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facbc000c00 tx=0x7facbc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.598+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7facc00aef40 con 0x7facc015a1f0 2023-11-28T11:09:17.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.777+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7facc00aef40 con 0x7facc015a1f0 2023-11-28T11:09:17.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.778+0000 7facb67fc700 1 -- 172.21.15.83:0/2815285992 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7facc00aef40 con 0x7facc015a1f0 2023-11-28T11:09:17.778 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:17.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.779+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faca40682f0 msgr2=0x7faca406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.779+0000 7facc7b54700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faca40682f0 0x7faca406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facbc000c00 tx=0x7facbc009040 comp rx=0 tx=0).stop 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.779+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 msgr2=0x7facc00c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.779+0000 7facc7b54700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 0x7facc00c6ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7facb00119c0 tx=0x7facb0011d80 comp rx=0 tx=0).stop 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 shutdown_connections 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facc00c7220 0x7facc00c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7facac004d90 0x7facc014a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faca40682f0 0x7faca406a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 --2- 172.21.15.83:0/2815285992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc015a1f0 0x7facc00c6ce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 >> 172.21.15.83:0/2815285992 conn(0x7facc00b2b60 msgr2=0x7facc015af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 shutdown_connections 2023-11-28T11:09:17.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.780+0000 7facc7b54700 1 -- 172.21.15.83:0/2815285992 wait complete. 2023-11-28T11:09:17.804 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:09:17.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:09:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.924+0000 7fd225189700 1 Processor -- start 2023-11-28T11:09:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.924+0000 7fd225189700 1 -- start start 2023-11-28T11:09:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.924+0000 7fd225189700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd2200b3070 0x7fd2200b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.924+0000 7fd225189700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22015f1d0 con 0x7fd22015a8e0 2023-11-28T11:09:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.924+0000 7fd225189700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22015eef0 con 0x7fd22015a510 2023-11-28T11:09:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.924+0000 7fd225189700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2200cd780 con 0x7fd2200b3070 2023-11-28T11:09:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21e59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd22015a8e0 0x7fd2200b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47590/0 (socket says 172.21.15.83:47590) 2023-11-28T11:09:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21e59c700 1 -- 172.21.15.83:0/69050042 learned_addr learned my addr 172.21.15.83:0/69050042 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21f59e700 1 --2- 172.21.15.83:0/69050042 >> v2:172.21.15.90:3300/0 conn(0x7fd2200b3070 0x7fd2200b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21dd9b700 1 -- 172.21.15.83:0/69050042 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258250554 0 0) 0x7fd22015f1d0 con 0x7fd22015a8e0 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21dd9b700 1 -- 172.21.15.83:0/69050042 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2080035b0 con 0x7fd22015a8e0 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21f59e700 1 -- 172.21.15.83:0/69050042 >> v1:172.21.15.90:6789/0 conn(0x7fd22015a510 legacy=0x7fd2200cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21f59e700 1 -- 172.21.15.83:0/69050042 >> v1:172.21.15.83:6789/0 conn(0x7fd22015a8e0 legacy=0x7fd2200b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21f59e700 1 -- 172.21.15.83:0/69050042 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200cedd0 con 0x7fd2200b3070 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21f59e700 1 --2- 172.21.15.83:0/69050042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2200b3070 0x7fd2200b3d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd218009d90 tx=0x7fd21800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca397eda608bda95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21dd9b700 1 -- 172.21.15.83:0/69050042 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800e070 con 0x7fd2200b3070 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21dd9b700 1 -- 172.21.15.83:0/69050042 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21800bb20 con 0x7fd2200b3070 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.925+0000 7fd21dd9b700 1 -- 172.21.15.83:0/69050042 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218012430 con 0x7fd2200b3070 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 -- 172.21.15.83:0/69050042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2200b3070 msgr2=0x7fd2200b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 --2- 172.21.15.83:0/69050042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2200b3070 0x7fd2200b3d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd218009d90 tx=0x7fd21800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 -- 172.21.15.83:0/69050042 shutdown_connections 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 --2- 172.21.15.83:0/69050042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2200b3070 0x7fd2200b3d30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 -- 172.21.15.83:0/69050042 >> 172.21.15.83:0/69050042 conn(0x7fd2200cd960 msgr2=0x7fd2200cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 -- 172.21.15.83:0/69050042 shutdown_connections 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.926+0000 7fd225189700 1 -- 172.21.15.83:0/69050042 wait complete. 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 Processor -- start 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 -- start start 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 0x7fd2201f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd22015a510 0x7fd22006c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd22015a8e0 0x7fd22006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201f2eb0 con 0x7fd2200b3070 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2200b6890 con 0x7fd22015a510 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd225189700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201f1bc0 con 0x7fd22015a8e0 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd21ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 0x7fd2201f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd21ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 0x7fd2201f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49736/0 (socket says 172.21.15.83:49736) 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.928+0000 7fd21ed9d700 1 -- 172.21.15.83:0/3697494264 learned_addr learned my addr 172.21.15.83:0/3697494264 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21ed9d700 1 -- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd22015a8e0 msgr2=0x7fd22006cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21e59c700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd22015a510 0x7fd22006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21f59e700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd22015a8e0 0x7fd22006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21ed9d700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd22015a8e0 0x7fd22006cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21ed9d700 1 -- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd22015a510 msgr2=0x7fd22006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21ed9d700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd22015a510 0x7fd22006c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21ed9d700 1 -- 172.21.15.83:0/3697494264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218009a70 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd21ed9d700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 0x7fd2201f2010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c510 tx=0x7fd21000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100052d0 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd210005430 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2200cf390 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100038c0 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.929+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2200cf0b0 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.930+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2200cffc0 con 0x7fd2200b3070 2023-11-28T11:09:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.930+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd210019030 con 0x7fd2200b3070 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.931+0000 7fd2077fe700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd20c0682f0 0x7fd20c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.931+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd2100852c0 con 0x7fd2200b3070 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.931+0000 7fd21e59c700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd20c0682f0 0x7fd20c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.931+0000 7fd21e59c700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd20c0682f0 0x7fd20c06a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2200b63a0 tx=0x7fd214009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:17.935+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd210051830 con 0x7fd2200b3070 2023-11-28T11:09:18.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.197+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fd220074320 con 0x7fd2200b3070 2023-11-28T11:09:18.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.989+0000 7fd2077fe700 1 -- 172.21.15.83:0/3697494264 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v55) v1 ==== 145+0+0 (secure 0 0 0) 0x7fd210054e50 con 0x7fd2200b3070 2023-11-28T11:09:18.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.991+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd20c0682f0 msgr2=0x7fd20c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.991+0000 7fd225189700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd20c0682f0 0x7fd20c06a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2200b63a0 tx=0x7fd214009040 comp rx=0 tx=0).stop 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.991+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 msgr2=0x7fd2201f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.991+0000 7fd225189700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 0x7fd2201f2010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c510 tx=0x7fd21000c8d0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 shutdown_connections 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd22015a8e0 0x7fd22006cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd20c0682f0 0x7fd20c06a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2200b3070 0x7fd2201f2010 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 --2- 172.21.15.83:0/3697494264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd22015a510 0x7fd22006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 >> 172.21.15.83:0/3697494264 conn(0x7fd2200cd960 msgr2=0x7fd2200cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:18.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 shutdown_connections 2023-11-28T11:09:18.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:18.992+0000 7fd225189700 1 -- 172.21.15.83:0/3697494264 wait complete. 2023-11-28T11:09:19.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:09:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.132+0000 7f8f02dfe700 1 Processor -- start 2023-11-28T11:09:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.132+0000 7f8f02dfe700 1 -- start start 2023-11-28T11:09:19.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f02dfe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8efc15c890 0x7f8efc0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f02dfe700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8efc0b4bb0 0x7f8efc06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f02dfe700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc06d4a0 con 0x7f8efc06e1a0 2023-11-28T11:09:19.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f02dfe700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc06e620 con 0x7f8efc15c890 2023-11-28T11:09:19.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f02dfe700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc0b5020 con 0x7f8efc0b4bb0 2023-11-28T11:09:19.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8efc15c890 0x7f8efc0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8efc15c890 0x7f8efc0aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49744/0 (socket says 172.21.15.83:49744) 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 -- 172.21.15.83:0/3247206655 learned_addr learned my addr 172.21.15.83:0/3247206655 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8efbfff700 1 --2- 172.21.15.83:0/3247206655 >> v2:172.21.15.90:3300/0 conn(0x7f8efc0b4bb0 0x7f8efc06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 -- 172.21.15.83:0/3247206655 >> v2:172.21.15.90:3300/0 conn(0x7f8efc0b4bb0 msgr2=0x7f8efc06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 --2- 172.21.15.83:0/3247206655 >> v2:172.21.15.90:3300/0 conn(0x7f8efc0b4bb0 0x7f8efc06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 -- 172.21.15.83:0/3247206655 >> v1:172.21.15.90:6789/0 conn(0x7f8efc06e1a0 legacy=0x7f8efc1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.133+0000 7f8f00b9a700 1 -- 172.21.15.83:0/3247206655 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc06c160 con 0x7f8efc15c890 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f00b9a700 1 --2- 172.21.15.83:0/3247206655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc15c890 0x7f8efc0aee60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8efc071890 tx=0x7f8ef0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e33d74682271793 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8efb7fe700 1 -- 172.21.15.83:0/3247206655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0005e30 con 0x7f8efc15c890 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8efb7fe700 1 -- 172.21.15.83:0/3247206655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0006030 con 0x7f8efc15c890 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8efb7fe700 1 -- 172.21.15.83:0/3247206655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00063e0 con 0x7f8efc15c890 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 -- 172.21.15.83:0/3247206655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc15c890 msgr2=0x7f8efc0aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/3247206655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc15c890 0x7f8efc0aee60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8efc071890 tx=0x7f8ef0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 -- 172.21.15.83:0/3247206655 shutdown_connections 2023-11-28T11:09:19.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/3247206655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc15c890 0x7f8efc0aee60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/3247206655 >> v2:172.21.15.90:3300/0 conn(0x7f8efc0b4bb0 0x7f8efc06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 -- 172.21.15.83:0/3247206655 >> 172.21.15.83:0/3247206655 conn(0x7f8efc0b2b60 msgr2=0x7f8efc0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 -- 172.21.15.83:0/3247206655 shutdown_connections 2023-11-28T11:09:19.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.134+0000 7f8f02dfe700 1 -- 172.21.15.83:0/3247206655 wait complete. 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 Processor -- start 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 -- start start 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 0x7f8efc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8efc0b4bb0 0x7f8efc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8efc15c890 0x7f8efc0c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0c86b0 con 0x7f8efc06e1a0 2023-11-28T11:09:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0c65e0 con 0x7f8efc0b4bb0 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.136+0000 7f8f02dfe700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0c4350 con 0x7f8efc15c890 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 0x7f8efc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 0x7f8efc155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49750/0 (socket says 172.21.15.83:49750) 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 -- 172.21.15.83:0/2795677859 learned_addr learned my addr 172.21.15.83:0/2795677859 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 -- 172.21.15.83:0/2795677859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8efc15c890 msgr2=0x7f8efc0c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8efc15c890 0x7f8efc0c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 -- 172.21.15.83:0/2795677859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8efc0b4bb0 msgr2=0x7f8efc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8efc0b4bb0 0x7f8efc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 -- 172.21.15.83:0/2795677859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0009500 con 0x7f8efc06e1a0 2023-11-28T11:09:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f00b9a700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 0x7f8efc155cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006000 tx=0x7f8ef0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0025430 con 0x7f8efc06e1a0 2023-11-28T11:09:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0020070 con 0x7f8efc06e1a0 2023-11-28T11:09:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc06c400 con 0x7f8efc06e1a0 2023-11-28T11:09:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00257d0 con 0x7f8efc06e1a0 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.137+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc0b5ae0 con 0x7f8efc06e1a0 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.138+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc0b5240 con 0x7f8efc06e1a0 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.138+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ef002b020 con 0x7f8efc06e1a0 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.139+0000 7f8ef97fa700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8ee0068320 0x7f8ee006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.139+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8ef0087a20 con 0x7f8efc06e1a0 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.139+0000 7f8efbfff700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8ee0068320 0x7f8ee006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.139+0000 7f8efbfff700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8ee0068320 0x7f8ee006a7d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b59b0 tx=0x7f8ee8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.143+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ef0053f10 con 0x7f8efc06e1a0 2023-11-28T11:09:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.324+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8efc0742b0 con 0x7f8efc06e1a0 2023-11-28T11:09:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.327+0000 7f8ef97fa700 1 -- 172.21.15.83:0/2795677859 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ef0057530 con 0x7f8efc06e1a0 2023-11-28T11:09:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8ee0068320 msgr2=0x7f8ee006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8ee0068320 0x7f8ee006a7d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b59b0 tx=0x7f8ee8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 msgr2=0x7f8efc155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 0x7f8efc155cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006000 tx=0x7f8ef0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 shutdown_connections 2023-11-28T11:09:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8efc15c890 0x7f8efc0c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8ee0068320 0x7f8ee006a7d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8efc06e1a0 0x7f8efc155cc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 --2- 172.21.15.83:0/2795677859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8efc0b4bb0 0x7f8efc0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.329+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 >> 172.21.15.83:0/2795677859 conn(0x7f8efc0b2b60 msgr2=0x7f8efc06c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.330+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 shutdown_connections 2023-11-28T11:09:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.330+0000 7f8f02dfe700 1 -- 172.21.15.83:0/2795677859 wait complete. 2023-11-28T11:09:19.343 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:09:19.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:19.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:19.443 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:09:19.443 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:19.443 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:19.461 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:19.462 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:19.520 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:19.520 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:19.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:19.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.738+0000 7fa8cd59b700 1 Processor -- start 2023-11-28T11:09:19.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.738+0000 7fa8cd59b700 1 -- start start 2023-11-28T11:09:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8cd59b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8c815c8e0 0x7fa8c80d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8cd59b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c8160560 0x7fa8c81609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8cd59b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c80d09e0 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8cd59b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c8160f20 con 0x7fa8c815c8e0 2023-11-28T11:09:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8cd59b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c806bb90 con 0x7fa8c8160560 2023-11-28T11:09:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8c815c8e0 0x7fa8c80d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa8c80d1aa0 0x7fa8c80d1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47622/0 (socket says 172.21.15.83:47622) 2023-11-28T11:09:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c77fe700 1 -- 172.21.15.83:0/4071154138 learned_addr learned my addr 172.21.15.83:0/4071154138 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c67fc700 1 --2- 172.21.15.83:0/4071154138 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c8160560 0x7fa8c81609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/4071154138 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c8160560 msgr2=0x7fa8c81609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c6ffd700 1 --2- 172.21.15.83:0/4071154138 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c8160560 0x7fa8c81609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/4071154138 >> v1:172.21.15.83:6789/0 conn(0x7fa8c80d1aa0 legacy=0x7fa8c80d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/4071154138 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c806b8b0 con 0x7fa8c815c8e0 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c5ffb700 1 -- 172.21.15.83:0/4071154138 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368190558 0 0) 0x7fa8c0001490 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c6ffd700 1 --2- 172.21.15.83:0/4071154138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c815c8e0 0x7fa8c80d04a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009840 tx=0x7fa8b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7d097cf60213dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c5ffb700 1 -- 172.21.15.83:0/4071154138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8011830 con 0x7fa8c815c8e0 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c5ffb700 1 -- 172.21.15.83:0/4071154138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b8011990 con 0x7fa8c815c8e0 2023-11-28T11:09:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.739+0000 7fa8c5ffb700 1 -- 172.21.15.83:0/4071154138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8011d20 con 0x7fa8c815c8e0 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 -- 172.21.15.83:0/4071154138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c815c8e0 msgr2=0x7fa8c80d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/4071154138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c815c8e0 0x7fa8c80d04a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009840 tx=0x7fa8b8009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 -- 172.21.15.83:0/4071154138 shutdown_connections 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/4071154138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c815c8e0 0x7fa8c80d04a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/4071154138 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c8160560 0x7fa8c81609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 -- 172.21.15.83:0/4071154138 >> 172.21.15.83:0/4071154138 conn(0x7fa8c80d3cb0 msgr2=0x7fa8c80cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 -- 172.21.15.83:0/4071154138 shutdown_connections 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.740+0000 7fa8cd59b700 1 -- 172.21.15.83:0/4071154138 wait complete. 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.742+0000 7fa8cd59b700 1 Processor -- start 2023-11-28T11:09:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.742+0000 7fa8cd59b700 1 -- start start 2023-11-28T11:09:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 0x7fa8c815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c815c8e0 0x7fa8c80cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8c8160560 0x7fa8c80cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81f6970 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80b8a10 con 0x7fa8c8160560 2023-11-28T11:09:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c815fb00 con 0x7fa8c815c8e0 2023-11-28T11:09:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 0x7fa8c815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 0x7fa8c815f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49764/0 (socket says 172.21.15.83:49764) 2023-11-28T11:09:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/272110831 learned_addr learned my addr 172.21.15.83:0/272110831 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c67fc700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c815c8e0 0x7fa8c80cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c77fe700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8c8160560 0x7fa8c80cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c815c8e0 msgr2=0x7fa8c80cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c815c8e0 0x7fa8c80cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8c8160560 msgr2=0x7fa8c80cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8c8160560 0x7fa8c80cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 -- 172.21.15.83:0/272110831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b80094a0 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c77fe700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8c8160560 0x7fa8c80cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8c6ffd700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 0x7fa8c815f5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80090b0 tx=0x7fa8b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80038a0 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b801f070 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c80d12a0 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003c20 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.743+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c80d0fc0 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.744+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c80d1580 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.745+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa8b800f050 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.745+0000 7fa8af7fe700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8b40682f0 0x7fa8b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.745+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa8b8021070 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.745+0000 7fa8c67fc700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8b40682f0 0x7fa8b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.746+0000 7fa8c67fc700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8b40682f0 0x7fa8b406a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80b3b00 tx=0x7fa8bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.749+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8b8053f20 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.936+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c8074320 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.937+0000 7fa8af7fe700 1 -- 172.21.15.83:0/272110831 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fa8b8057540 con 0x7fa8c80d1aa0 2023-11-28T11:09:19.938 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:19.938 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:19.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8b40682f0 msgr2=0x7fa8b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8b40682f0 0x7fa8b406a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80b3b00 tx=0x7fa8bc008040 comp rx=0 tx=0).stop 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 msgr2=0x7fa8c815f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 0x7fa8c815f5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80090b0 tx=0x7fa8b8004750 comp rx=0 tx=0).stop 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 shutdown_connections 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8c8160560 0x7fa8c80cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8b40682f0 0x7fa8b406a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c80d1aa0 0x7fa8c815f5c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 --2- 172.21.15.83:0/272110831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8c815c8e0 0x7fa8c80cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 >> 172.21.15.83:0/272110831 conn(0x7fa8c80d3cb0 msgr2=0x7fa8c80b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 shutdown_connections 2023-11-28T11:09:19.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:19.939+0000 7fa8cd59b700 1 -- 172.21.15.83:0/272110831 wait complete. 2023-11-28T11:09:19.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:19.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:20.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.075+0000 7f64ab3c5700 1 Processor -- start 2023-11-28T11:09:20.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.075+0000 7f64ab3c5700 1 -- start start 2023-11-28T11:09:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64ab3c5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40af040 0x7f64a40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64ab3c5700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f64a406c500 0x7f64a406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64ab3c5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a40cd6a0 con 0x7f64a406f440 2023-11-28T11:09:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64ab3c5700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a40cce00 con 0x7f64a406c500 2023-11-28T11:09:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64ab3c5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40cd0e0 con 0x7f64a40af040 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64a9962700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f64a406f440 0x7f64a40b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47636/0 (socket says 172.21.15.83:47636) 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.076+0000 7f64a9962700 1 -- 172.21.15.83:0/1990244244 learned_addr learned my addr 172.21.15.83:0/1990244244 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a8960700 1 --2- 172.21.15.83:0/1990244244 >> v2:172.21.15.90:3300/0 conn(0x7f64a406c500 0x7f64a406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9161700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40af040 0x7f64a40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901120807 0 0) 0x7f64a40cd6a0 con 0x7f64a406f440 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64900035b0 con 0x7f64a406f440 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090190461 0 0) 0x7f64900035b0 con 0x7f64a406f440 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40af040 msgr2=0x7f64a40cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40af040 0x7f64a40cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 >> v2:172.21.15.90:3300/0 conn(0x7f64a406c500 msgr2=0x7f64a406ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 --2- 172.21.15.83:0/1990244244 >> v2:172.21.15.90:3300/0 conn(0x7f64a406c500 0x7f64a406ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40cd3c0 con 0x7f64a406f440 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f649c0019f0 con 0x7f64a406f440 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 >> v1:172.21.15.83:6789/0 conn(0x7f64a406f440 legacy=0x7f64a40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6490003a00 0x7f6490003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64900043e0 0x7f64900048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f649000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40cce00 con 0x7f6490004f20 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40cd0e0 con 0x7f64900043e0 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649000bf50 con 0x7f6490003a00 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9161700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40af040 0x7f64a40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a8960700 1 --2- 172.21.15.83:0/1990244244 >> v2:172.21.15.90:3300/0 conn(0x7f64a406c500 0x7f64a406ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f649000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9161700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6490003a00 0x7f6490003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a8960700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64900043e0 0x7f64900048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 -- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6490003a00 msgr2=0x7f6490003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6490003a00 0x7f6490003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 -- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64900043e0 msgr2=0x7f64900048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64900043e0 0x7f64900048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 -- 172.21.15.83:0/1990244244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900035b0 con 0x7f6490004f20 2023-11-28T11:09:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.077+0000 7f64a9962700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f649000ba10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f649c00a440 tx=0x7f6490005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e091b7eb7b3fe1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c010690 con 0x7f6490004f20 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c010810 con 0x7f6490004f20 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f649bfff700 1 -- 172.21.15.83:0/1990244244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c010bc0 con 0x7f6490004f20 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1990244244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 msgr2=0x7f649000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1990244244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f649000ba10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f649c00a440 tx=0x7f6490005880 comp rx=0 tx=0).stop 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64a9962700 1 -- 172.21.15.83:0/1990244244 reap_dead start 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1990244244 shutdown_connections 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1990244244 >> 172.21.15.83:0/1990244244 conn(0x7f64a40b2c10 msgr2=0x7f64a40b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1990244244 shutdown_connections 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.078+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1990244244 wait complete. 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.079+0000 7f64ab3c5700 1 Processor -- start 2023-11-28T11:09:20.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 -- start start 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f64a4154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64a40c4470 0x7f64a40c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40c5060 0x7f64a415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40c3fe0 con 0x7f6490004f20 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4155450 con 0x7f64a40c4470 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64ab3c5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40c5600 con 0x7f64a40c5060 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64a9161700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f64a4154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64a9161700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f64a4154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49784/0 (socket says 172.21.15.83:49784) 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64a9161700 1 -- 172.21.15.83:0/1456803752 learned_addr learned my addr 172.21.15.83:0/1456803752 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64a8960700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64a40c4470 0x7f64a40c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.080+0000 7f64a9962700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40c5060 0x7f64a415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64a9161700 1 -- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40c5060 msgr2=0x7f64a415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64a9161700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40c5060 0x7f64a415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64a9161700 1 -- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64a40c4470 msgr2=0x7f64a40c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64a9161700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64a40c4470 0x7f64a40c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64a9161700 1 -- 172.21.15.83:0/1456803752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a406f890 con 0x7f6490004f20 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64a9161700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f64a4154f10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6494011800 tx=0x7f6494011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494015870 con 0x7f6490004f20 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940159d0 con 0x7f6490004f20 2023-11-28T11:09:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a40ca420 con 0x7f6490004f20 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494015d80 con 0x7f6490004f20 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.081+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a40af410 con 0x7f6490004f20 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.082+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a406e580 con 0x7f6490004f20 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.083+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6494012040 con 0x7f6490004f20 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.083+0000 7f6499ffb700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64880682f0 0x7f648806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.083+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f64940866b0 con 0x7f6490004f20 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.083+0000 7f64a8960700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64880682f0 0x7f648806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.083+0000 7f64a8960700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64880682f0 0x7f648806a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64a40b23f0 tx=0x7f64a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.087+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6494052c20 con 0x7f6490004f20 2023-11-28T11:09:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.267+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f64a40c37b0 con 0x7f6490004f20 2023-11-28T11:09:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.268+0000 7f6499ffb700 1 -- 172.21.15.83:0/1456803752 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6494056240 con 0x7f6490004f20 2023-11-28T11:09:20.271 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.269+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64880682f0 msgr2=0x7f648806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.269+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64880682f0 0x7f648806a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64a40b23f0 tx=0x7f64a0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.269+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 msgr2=0x7f64a4154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.269+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f64a4154f10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6494011800 tx=0x7f6494011bc0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 shutdown_connections 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64a40c5060 0x7f64a415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64880682f0 0x7f648806a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490004f20 0x7f64a4154f10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 --2- 172.21.15.83:0/1456803752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64a40c4470 0x7f64a40c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 >> 172.21.15.83:0/1456803752 conn(0x7f64a40b2c10 msgr2=0x7f64a40cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 shutdown_connections 2023-11-28T11:09:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.270+0000 7f64ab3c5700 1 -- 172.21.15.83:0/1456803752 wait complete. 2023-11-28T11:09:20.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:20.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.404+0000 7f180cab7700 1 Processor -- start 2023-11-28T11:09:20.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.404+0000 7f180cab7700 1 -- start start 2023-11-28T11:09:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.405+0000 7f180cab7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f180815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.405+0000 7f180cab7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f180815b3b0 0x7f18080d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.405+0000 7f180cab7700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1808151a30 con 0x7f180815b820 2023-11-28T11:09:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.405+0000 7f180cab7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18080cb460 con 0x7f180815b3b0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.405+0000 7f180cab7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080cb740 con 0x7f180815e5c0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.405+0000 7f1805d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f180815b3b0 0x7f18080d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f180815b3b0 0x7f18080d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49794/0 (socket says 172.21.15.83:49794) 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 -- 172.21.15.83:0/2384904409 learned_addr learned my addr 172.21.15.83:0/2384904409 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f180659c700 1 --2- 172.21.15.83:0/2384904409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f180815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 -- 172.21.15.83:0/2384904409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 msgr2=0x7f180815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 --2- 172.21.15.83:0/2384904409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f180815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 -- 172.21.15.83:0/2384904409 >> v1:172.21.15.90:6789/0 conn(0x7f180815b820 legacy=0x7f18080ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 -- 172.21.15.83:0/2384904409 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180815abb0 con 0x7f180815b3b0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f1805d9b700 1 --2- 172.21.15.83:0/2384904409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b3b0 0x7f18080d0ae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f17f80098f0 tx=0x7f17f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342c246d2b670752 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f180559a700 1 -- 172.21.15.83:0/2384904409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f800b8a0 con 0x7f180815b3b0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f180559a700 1 -- 172.21.15.83:0/2384904409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f800ba00 con 0x7f180815b3b0 2023-11-28T11:09:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.406+0000 7f180559a700 1 -- 172.21.15.83:0/2384904409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f800bd90 con 0x7f180815b3b0 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 -- 172.21.15.83:0/2384904409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b3b0 msgr2=0x7f18080d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 --2- 172.21.15.83:0/2384904409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b3b0 0x7f18080d0ae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f17f80098f0 tx=0x7f17f8009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 -- 172.21.15.83:0/2384904409 shutdown_connections 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 --2- 172.21.15.83:0/2384904409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f180815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 --2- 172.21.15.83:0/2384904409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b3b0 0x7f18080d0ae0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 -- 172.21.15.83:0/2384904409 >> 172.21.15.83:0/2384904409 conn(0x7f180806ea50 msgr2=0x7f1808070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 -- 172.21.15.83:0/2384904409 shutdown_connections 2023-11-28T11:09:20.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.407+0000 7f180cab7700 1 -- 172.21.15.83:0/2384904409 wait complete. 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.408+0000 7f180cab7700 1 Processor -- start 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.408+0000 7f180cab7700 1 -- start start 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180cab7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 0x7f18080b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180cab7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f1808150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180cab7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1808150e70 0x7f18081e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180cab7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1808151bc0 con 0x7f180815b820 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180cab7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18081512e0 con 0x7f1808150e70 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180cab7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18081e62b0 con 0x7f180815e5c0 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 0x7f18080b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 0x7f18080b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49798/0 (socket says 172.21.15.83:49798) 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 -- 172.21.15.83:0/1692916746 learned_addr learned my addr 172.21.15.83:0/1692916746 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f1805d9b700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f1808150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f1806d9d700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1808150e70 0x7f18081e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 -- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 msgr2=0x7f1808150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f1808150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 -- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1808150e70 msgr2=0x7f18081e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1808150e70 0x7f18081e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 -- 172.21.15.83:0/1692916746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f8009550 con 0x7f180815b820 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.409+0000 7f180659c700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 0x7f18080b10f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e840 tx=0x7f17fc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.410+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00c070 con 0x7f180815b820 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.410+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc013610 con 0x7f180815b820 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.410+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180815eed0 con 0x7f180815b820 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.410+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0139c0 con 0x7f180815b820 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.410+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180815ae90 con 0x7f180815b820 2023-11-28T11:09:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.411+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18080cba20 con 0x7f180815b820 2023-11-28T11:09:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.411+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f17fc013b20 con 0x7f180815b820 2023-11-28T11:09:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.411+0000 7f17f6ffd700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f17e80682f0 0x7f17e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.412+0000 7f1805d9b700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f17e80682f0 0x7f17e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.412+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f17fc085c60 con 0x7f180815b820 2023-11-28T11:09:20.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.412+0000 7f1805d9b700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f17e80682f0 0x7f17e806a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f17f8014040 tx=0x7f17f8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.416+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f17fc052170 con 0x7f180815b820 2023-11-28T11:09:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.595+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18080742b0 con 0x7f180815b820 2023-11-28T11:09:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.596+0000 7f17f6ffd700 1 -- 172.21.15.83:0/1692916746 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17fc0170a0 con 0x7f180815b820 2023-11-28T11:09:20.596 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:20.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f17e80682f0 msgr2=0x7f17e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f17e80682f0 0x7f17e806a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f17f8014040 tx=0x7f17f8008030 comp rx=0 tx=0).stop 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 msgr2=0x7f18080b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 0x7f18080b10f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e840 tx=0x7f17fc00eb50 comp rx=0 tx=0).stop 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 shutdown_connections 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1808150e70 0x7f18081e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f17e80682f0 0x7f17e806a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180815b820 0x7f18080b10f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 --2- 172.21.15.83:0/1692916746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f180815e5c0 0x7f1808150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 >> 172.21.15.83:0/1692916746 conn(0x7f180815b3b0 msgr2=0x7f1808070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 shutdown_connections 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.598+0000 7f180cab7700 1 -- 172.21.15.83:0/1692916746 wait complete. 2023-11-28T11:09:20.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:20.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.732+0000 7f3318123700 1 Processor -- start 2023-11-28T11:09:20.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.732+0000 7f3318123700 1 -- start start 2023-11-28T11:09:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f3318123700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33100ce1b0 0x7f331015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f3318123700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f33100d0ae0 0x7f33100d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f3318123700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3310151a30 con 0x7f331015e5c0 2023-11-28T11:09:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f3318123700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100d1440 con 0x7f33100ce1b0 2023-11-28T11:09:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f3318123700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100cb460 con 0x7f33100d0ae0 2023-11-28T11:09:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33100ce1b0 0x7f331015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33100ce1b0 0x7f331015b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49808/0 (socket says 172.21.15.83:49808) 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 -- 172.21.15.83:0/2556221906 learned_addr learned my addr 172.21.15.83:0/2556221906 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33166c0700 1 --2- 172.21.15.83:0/2556221906 >> v2:172.21.15.90:3300/0 conn(0x7f33100d0ae0 0x7f33100d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 -- 172.21.15.83:0/2556221906 >> v2:172.21.15.90:3300/0 conn(0x7f33100d0ae0 msgr2=0x7f33100d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 --2- 172.21.15.83:0/2556221906 >> v2:172.21.15.90:3300/0 conn(0x7f33100d0ae0 0x7f33100d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 -- 172.21.15.83:0/2556221906 >> v1:172.21.15.90:6789/0 conn(0x7f331015e5c0 legacy=0x7f331015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.733+0000 7f33156be700 1 -- 172.21.15.83:0/2556221906 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100cb740 con 0x7f33100ce1b0 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f33156be700 1 --2- 172.21.15.83:0/2556221906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100ce1b0 0x7f331015b3b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f33100cb2c0 tx=0x7f330c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f62a4f84cf0a43e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3314ebd700 1 -- 172.21.15.83:0/2556221906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c004230 con 0x7f33100ce1b0 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3314ebd700 1 -- 172.21.15.83:0/2556221906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c004390 con 0x7f33100ce1b0 2023-11-28T11:09:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3314ebd700 1 -- 172.21.15.83:0/2556221906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c0036a0 con 0x7f33100ce1b0 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 -- 172.21.15.83:0/2556221906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100ce1b0 msgr2=0x7f331015b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 --2- 172.21.15.83:0/2556221906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100ce1b0 0x7f331015b3b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f33100cb2c0 tx=0x7f330c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 -- 172.21.15.83:0/2556221906 shutdown_connections 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 --2- 172.21.15.83:0/2556221906 >> v2:172.21.15.90:3300/0 conn(0x7f33100d0ae0 0x7f33100d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 --2- 172.21.15.83:0/2556221906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100ce1b0 0x7f331015b3b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 -- 172.21.15.83:0/2556221906 >> 172.21.15.83:0/2556221906 conn(0x7f331006ea50 msgr2=0x7f3310070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 -- 172.21.15.83:0/2556221906 shutdown_connections 2023-11-28T11:09:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.734+0000 7f3318123700 1 -- 172.21.15.83:0/2556221906 wait complete. 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 Processor -- start 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 -- start start 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 0x7f331014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f331015e5c0 0x7f33100b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3310150b20 0x7f331015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33100b6a00 con 0x7f33100d0ae0 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331014e240 con 0x7f3310150b20 2023-11-28T11:09:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3318123700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33100b4410 con 0x7f331015e5c0 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3315ebf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 0x7f331014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3315ebf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 0x7f331014dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49812/0 (socket says 172.21.15.83:49812) 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.736+0000 7f3315ebf700 1 -- 172.21.15.83:0/1183375403 learned_addr learned my addr 172.21.15.83:0/1183375403 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f33166c0700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3310150b20 0x7f331015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f33156be700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f331015e5c0 0x7f33100b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3315ebf700 1 -- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f331015e5c0 msgr2=0x7f33100b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3315ebf700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f331015e5c0 0x7f33100b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3315ebf700 1 -- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3310150b20 msgr2=0x7f331015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3315ebf700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3310150b20 0x7f331015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3315ebf700 1 -- 172.21.15.83:0/1183375403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c009550 con 0x7f33100d0ae0 2023-11-28T11:09:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3315ebf700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 0x7f331014dd00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f330000b4a0 tx=0x7f330000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300011830 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3300011990 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300011d40 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331015ae90 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.737+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331015abb0 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.738+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331015b8f0 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.738+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f330000f050 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.739+0000 7f33067fc700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f32f40682f0 0x7f32f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.739+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3300085ae0 con 0x7f33100d0ae0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.739+0000 7f33156be700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f32f40682f0 0x7f32f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.739+0000 7f33156be700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f32f40682f0 0x7f32f406a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f330c009130 tx=0x7f330c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.743+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3300052050 con 0x7f33100d0ae0 2023-11-28T11:09:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.925+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33100742b0 con 0x7f33100d0ae0 2023-11-28T11:09:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.925+0000 7f33067fc700 1 -- 172.21.15.83:0/1183375403 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3300055670 con 0x7f33100d0ae0 2023-11-28T11:09:20.926 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.927+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f32f40682f0 msgr2=0x7f32f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.927+0000 7f3318123700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f32f40682f0 0x7f32f406a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f330c009130 tx=0x7f330c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.927+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 msgr2=0x7f331014dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.927+0000 7f3318123700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 0x7f331014dd00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f330000b4a0 tx=0x7f330000b860 comp rx=0 tx=0).stop 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 shutdown_connections 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3310150b20 0x7f331015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f32f40682f0 0x7f32f406a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33100d0ae0 0x7f331014dd00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 --2- 172.21.15.83:0/1183375403 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f331015e5c0 0x7f33100b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 >> 172.21.15.83:0/1183375403 conn(0x7f33100ce1b0 msgr2=0x7f3310151100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 shutdown_connections 2023-11-28T11:09:20.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:20.928+0000 7f3318123700 1 -- 172.21.15.83:0/1183375403 wait complete. 2023-11-28T11:09:20.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.064+0000 7f5a22db1700 1 Processor -- start 2023-11-28T11:09:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.064+0000 7f5a22db1700 1 -- start start 2023-11-28T11:09:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a22db1700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5a1c06e370 0x7f5a1c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a22db1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c06d4a0 con 0x7f5a1c0b4bb0 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a22db1700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c0b5ae0 con 0x7f5a1c15c890 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a22db1700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c0b5050 con 0x7f5a1c06e370 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a2134e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5a1c0b4bb0 0x7f5a1c06b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47684/0 (socket says 172.21.15.83:47684) 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a2134e700 1 -- 172.21.15.83:0/3387057019 learned_addr learned my addr 172.21.15.83:0/3387057019 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a1bfff700 1 --2- 172.21.15.83:0/3387057019 >> v2:172.21.15.90:3300/0 conn(0x7f5a1c06e370 0x7f5a1c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284040130 0 0) 0x7f5a1c06d4a0 con 0x7f5a1c0b4bb0 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c0035b0 con 0x7f5a1c0b4bb0 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909414628 0 0) 0x7f5a0c0035b0 con 0x7f5a1c0b4bb0 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 >> v2:172.21.15.90:3300/0 conn(0x7f5a1c06e370 msgr2=0x7f5a1c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.065+0000 7f5a1b7fe700 1 --2- 172.21.15.83:0/3387057019 >> v2:172.21.15.90:3300/0 conn(0x7f5a1c06e370 0x7f5a1c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 >> v1:172.21.15.90:6789/0 conn(0x7f5a1c15c890 legacy=0x7f5a1c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c06c400 con 0x7f5a1c0b4bb0 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f5a140019f0 con 0x7f5a1c0b4bb0 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 >> v1:172.21.15.83:6789/0 conn(0x7f5a1c0b4bb0 legacy=0x7f5a1c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c003a00 0x7f5a0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c005200 0x7f5a0c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00c0e0 con 0x7f5a0c005200 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00c250 con 0x7f5a0c003a00 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00c410 con 0x7f5a0c004850 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c005200 0x7f5a0c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a20b4d700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c003a00 0x7f5a0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1bfff700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 -- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a0c004850 msgr2=0x7f5a0c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 -- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c003a00 msgr2=0x7f5a0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c003a00 0x7f5a0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 -- 172.21.15.83:0/3387057019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0035b0 con 0x7f5a0c005200 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1bfff700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 -- 172.21.15.83:0/3387057019 reap_dead start 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a2134e700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c005200 0x7f5a0c00ba10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400a830 tx=0x7f5a0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da1d28d3f876094 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400a2f0 con 0x7f5a0c005200 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a14012600 con 0x7f5a0c005200 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.066+0000 7f5a1b7fe700 1 -- 172.21.15.83:0/3387057019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140129b0 con 0x7f5a0c005200 2023-11-28T11:09:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 -- 172.21.15.83:0/3387057019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c005200 msgr2=0x7f5a0c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c005200 0x7f5a0c00ba10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400a830 tx=0x7f5a0c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 -- 172.21.15.83:0/3387057019 shutdown_connections 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3387057019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c005200 0x7f5a0c00ba10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 -- 172.21.15.83:0/3387057019 >> 172.21.15.83:0/3387057019 conn(0x7f5a1c0b2b60 msgr2=0x7f5a1c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 -- 172.21.15.83:0/3387057019 shutdown_connections 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.067+0000 7f5a22db1700 1 -- 172.21.15.83:0/3387057019 wait complete. 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.068+0000 7f5a22db1700 1 Processor -- start 2023-11-28T11:09:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.068+0000 7f5a22db1700 1 -- start start 2023-11-28T11:09:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a22db1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c005200 0x7f5a1c0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a22db1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 0x7f5a1c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a22db1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a1c1e69e0 0x7f5a1c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a22db1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0c7d00 con 0x7f5a1c0c7890 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a22db1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0cc560 con 0x7f5a0c005200 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a22db1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0cb6b0 con 0x7f5a1c1e69e0 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 0x7f5a1c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 0x7f5a1c0cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49834/0 (socket says 172.21.15.83:49834) 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 -- 172.21.15.83:0/3914401487 learned_addr learned my addr 172.21.15.83:0/3914401487 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a2134e700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a1c1e69e0 0x7f5a1c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a20b4d700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c005200 0x7f5a1c0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 -- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a1c1e69e0 msgr2=0x7f5a1c15a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a1c1e69e0 0x7f5a1c15a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 -- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c005200 msgr2=0x7f5a1c0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c005200 0x7f5a1c0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 -- 172.21.15.83:0/3914401487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0b5520 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.069+0000 7f5a1bfff700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 0x7f5a1c0cb170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5a080108c0 tx=0x7f5a08010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.070+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004020 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.070+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08004180 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.070+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c0b5240 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.070+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004510 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.070+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c0b5800 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.071+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c06ec60 con 0x7f5a1c0c7890 2023-11-28T11:09:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.071+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5a080036a0 con 0x7f5a1c0c7890 2023-11-28T11:09:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.071+0000 7f5a197fa700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a000682f0 0x7f5a0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.071+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a08085f20 con 0x7f5a1c0c7890 2023-11-28T11:09:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.071+0000 7f5a20b4d700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a000682f0 0x7f5a0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.072+0000 7f5a20b4d700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a000682f0 0x7f5a0006a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000c00 tx=0x7f5a10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.075+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5a08052430 con 0x7f5a1c0c7890 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.255+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a1c0b3870 con 0x7f5a1c0c7890 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.255+0000 7f5a197fa700 1 -- 172.21.15.83:0/3914401487 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a08023020 con 0x7f5a1c0c7890 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a000682f0 msgr2=0x7f5a0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a000682f0 0x7f5a0006a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000c00 tx=0x7f5a10009040 comp rx=0 tx=0).stop 2023-11-28T11:09:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 msgr2=0x7f5a1c0cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 0x7f5a1c0cb170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5a080108c0 tx=0x7f5a08010bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 shutdown_connections 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a1c1e69e0 0x7f5a1c15a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a0c005200 0x7f5a1c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a000682f0 0x7f5a0006a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 --2- 172.21.15.83:0/3914401487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a1c0c7890 0x7f5a1c0cb170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 >> 172.21.15.83:0/3914401487 conn(0x7f5a1c0b2b60 msgr2=0x7f5a1c06b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 shutdown_connections 2023-11-28T11:09:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.257+0000 7f5a22db1700 1 -- 172.21.15.83:0/3914401487 wait complete. 2023-11-28T11:09:21.271 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:21.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 Processor -- start 2023-11-28T11:09:21.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 -- start start 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2e3c06e370 0x7f2e3c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3c0b4bb0 0x7f2e3c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c06d4a0 con 0x7f2e3c15c890 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0b5050 con 0x7f2e3c0b4bb0 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.394+0000 7f2e42fa1700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0b5ae0 con 0x7f2e3c06e370 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3c0b4bb0 0x7f2e3c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e40d3d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e3c15c890 0x7f2e3c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47700/0 (socket says 172.21.15.83:47700) 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3c0b4bb0 0x7f2e3c06b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49844/0 (socket says 172.21.15.83:49844) 2023-11-28T11:09:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e40d3d700 1 -- 172.21.15.83:0/1469680361 learned_addr learned my addr 172.21.15.83:0/1469680361 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e3bfff700 1 --2- 172.21.15.83:0/1469680361 >> v2:172.21.15.90:3300/0 conn(0x7f2e3c06e370 0x7f2e3c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 -- 172.21.15.83:0/1469680361 >> v2:172.21.15.90:3300/0 conn(0x7f2e3c06e370 msgr2=0x7f2e3c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 --2- 172.21.15.83:0/1469680361 >> v2:172.21.15.90:3300/0 conn(0x7f2e3c06e370 0x7f2e3c0aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 -- 172.21.15.83:0/1469680361 >> v1:172.21.15.83:6789/0 conn(0x7f2e3c15c890 legacy=0x7f2e3c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e3b7fe700 1 -- 172.21.15.83:0/1469680361 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473905944 0 0) 0x7f2e3c06d4a0 con 0x7f2e3c15c890 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 -- 172.21.15.83:0/1469680361 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c06c400 con 0x7f2e3c0b4bb0 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e4153e700 1 --2- 172.21.15.83:0/1469680361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c0b4bb0 0x7f2e3c06b150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c071890 tx=0x7f2e3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63f9cc33bd5c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e3b7fe700 1 -- 172.21.15.83:0/1469680361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400e070 con 0x7f2e3c0b4bb0 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e3b7fe700 1 -- 172.21.15.83:0/1469680361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400bbd0 con 0x7f2e3c0b4bb0 2023-11-28T11:09:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.395+0000 7f2e3b7fe700 1 -- 172.21.15.83:0/1469680361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34012430 con 0x7f2e3c0b4bb0 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 -- 172.21.15.83:0/1469680361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c0b4bb0 msgr2=0x7f2e3c06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/1469680361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c0b4bb0 0x7f2e3c06b150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c071890 tx=0x7f2e3400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 -- 172.21.15.83:0/1469680361 shutdown_connections 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/1469680361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c0b4bb0 0x7f2e3c06b150 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/1469680361 >> v2:172.21.15.90:3300/0 conn(0x7f2e3c06e370 0x7f2e3c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 -- 172.21.15.83:0/1469680361 >> 172.21.15.83:0/1469680361 conn(0x7f2e3c0b2b60 msgr2=0x7f2e3c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 -- 172.21.15.83:0/1469680361 shutdown_connections 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.396+0000 7f2e42fa1700 1 -- 172.21.15.83:0/1469680361 wait complete. 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.397+0000 7f2e42fa1700 1 Processor -- start 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.397+0000 7f2e42fa1700 1 -- start start 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e42fa1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 0x7f2e3c0adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e42fa1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2e3c0b4bb0 0x7f2e3c0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e42fa1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e3c15c890 0x7f2e3c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e42fa1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c9ad0 con 0x7f2e3c06e370 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e42fa1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c8ec0 con 0x7f2e3c15c890 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e42fa1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c91a0 con 0x7f2e3c0b4bb0 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 0x7f2e3c0adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 0x7f2e3c0adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49846/0 (socket says 172.21.15.83:49846) 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 -- 172.21.15.83:0/2402268061 learned_addr learned my addr 172.21.15.83:0/2402268061 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e3bfff700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2e3c0b4bb0 0x7f2e3c0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e4153e700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e3c15c890 0x7f2e3c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 -- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2e3c0b4bb0 msgr2=0x7f2e3c0c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2e3c0b4bb0 0x7f2e3c0c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 -- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e3c15c890 msgr2=0x7f2e3c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e3c15c890 0x7f2e3c154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 -- 172.21.15.83:0/2402268061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34009a70 con 0x7f2e3c06e370 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e4153e700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e3c15c890 0x7f2e3c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.398+0000 7f2e40d3d700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 0x7f2e3c0adc50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c4a0 tx=0x7f2e3000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.399+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e300074e0 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.399+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30007640 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.399+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30005a70 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.399+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c0b5520 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.399+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c0b5240 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.400+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c0b5800 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.400+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2e30019030 con 0x7f2e3c06e370 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.400+0000 7f2e397fa700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e200682f0 0x7f2e2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.401+0000 7f2e3bfff700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e200682f0 0x7f2e2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.401+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2e30085270 con 0x7f2e3c06e370 2023-11-28T11:09:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.401+0000 7f2e3bfff700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e200682f0 0x7f2e2006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006fd0 tx=0x7f2e28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.405+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2e300517e0 con 0x7f2e3c06e370 2023-11-28T11:09:21.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.585+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2e3c0cde20 con 0x7f2e3c06e370 2023-11-28T11:09:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.585+0000 7f2e397fa700 1 -- 172.21.15.83:0/2402268061 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2e30017020 con 0x7f2e3c06e370 2023-11-28T11:09:21.587 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e200682f0 msgr2=0x7f2e2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e200682f0 0x7f2e2006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006fd0 tx=0x7f2e28008040 comp rx=0 tx=0).stop 2023-11-28T11:09:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 msgr2=0x7f2e3c0adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 0x7f2e3c0adc50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c4a0 tx=0x7f2e3000c7b0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 shutdown_connections 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e3c15c890 0x7f2e3c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2e200682f0 0x7f2e2006a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3c06e370 0x7f2e3c0adc50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 --2- 172.21.15.83:0/2402268061 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2e3c0b4bb0 0x7f2e3c0c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.587+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 >> 172.21.15.83:0/2402268061 conn(0x7f2e3c0b2b60 msgr2=0x7f2e3c0cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.588+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 shutdown_connections 2023-11-28T11:09:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.588+0000 7f2e42fa1700 1 -- 172.21.15.83:0/2402268061 wait complete. 2023-11-28T11:09:21.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.720+0000 7fe83f7e6700 1 Processor -- start 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 -- start start 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe838160150 0x7fe8380cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe83815d3b0 0x7fe8380ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8380cca00 con 0x7fe838160150 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe838160930 con 0x7fe83815d3b0 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.721+0000 7fe83f7e6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe838160c10 con 0x7fe8380cc0f0 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe838160150 0x7fe8380cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe838160150 0x7fe8380cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49854/0 (socket says 172.21.15.83:49854) 2023-11-28T11:09:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 -- 172.21.15.83:0/1164770958 learned_addr learned my addr 172.21.15.83:0/1164770958 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83dd83700 1 --2- 172.21.15.83:0/1164770958 >> v2:172.21.15.90:3300/0 conn(0x7fe83815d3b0 0x7fe8380ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83d582700 1 --2- 172.21.15.83:0/1164770958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 -- 172.21.15.83:0/1164770958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 msgr2=0x7fe8380cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 --2- 172.21.15.83:0/1164770958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 -- 172.21.15.83:0/1164770958 >> v2:172.21.15.90:3300/0 conn(0x7fe83815d3b0 msgr2=0x7fe8380ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 --2- 172.21.15.83:0/1164770958 >> v2:172.21.15.90:3300/0 conn(0x7fe83815d3b0 0x7fe8380ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 -- 172.21.15.83:0/1164770958 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838160ef0 con 0x7fe838160150 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe83cd81700 1 --2- 172.21.15.83:0/1164770958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe838160150 0x7fe8380cc4c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe8380c9940 tx=0x7fe834009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f975efdff5dac8c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe82ffff700 1 -- 172.21.15.83:0/1164770958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400b870 con 0x7fe838160150 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe82ffff700 1 -- 172.21.15.83:0/1164770958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400b9d0 con 0x7fe838160150 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.722+0000 7fe82ffff700 1 -- 172.21.15.83:0/1164770958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400bd60 con 0x7fe838160150 2023-11-28T11:09:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 -- 172.21.15.83:0/1164770958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe838160150 msgr2=0x7fe8380cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/1164770958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe838160150 0x7fe8380cc4c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe8380c9940 tx=0x7fe834009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 -- 172.21.15.83:0/1164770958 shutdown_connections 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/1164770958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/1164770958 >> v2:172.21.15.90:3300/0 conn(0x7fe83815d3b0 0x7fe8380ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/1164770958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe838160150 0x7fe8380cc4c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 -- 172.21.15.83:0/1164770958 >> 172.21.15.83:0/1164770958 conn(0x7fe8380b1090 msgr2=0x7fe8380b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 -- 172.21.15.83:0/1164770958 shutdown_connections 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.723+0000 7fe83f7e6700 1 -- 172.21.15.83:0/1164770958 wait complete. 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.724+0000 7fe83f7e6700 1 Processor -- start 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.724+0000 7fe83f7e6700 1 -- start start 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 0x7fe8380b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe838160150 0x7fe8380b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380b44e0 con 0x7fe83815d3b0 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380b47c0 con 0x7fe838160150 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380b4aa0 con 0x7fe8380cc0f0 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 0x7fe8380b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 0x7fe8380b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49862/0 (socket says 172.21.15.83:49862) 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 -- 172.21.15.83:0/386844112 learned_addr learned my addr 172.21.15.83:0/386844112 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83dd83700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe838160150 0x7fe8380b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83d582700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 -- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 msgr2=0x7fe8380c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 -- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe838160150 msgr2=0x7fe8380b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe838160150 0x7fe8380b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 -- 172.21.15.83:0/386844112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009550 con 0x7fe83815d3b0 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83cd81700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 0x7fe8380b3920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe834000c00 tx=0x7fe834004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340036a0 con 0x7fe83815d3b0 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834003800 con 0x7fe83815d3b0 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83815cb60 con 0x7fe83815d3b0 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.725+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834003b90 con 0x7fe83815d3b0 2023-11-28T11:09:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.726+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83815ce90 con 0x7fe83815d3b0 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.726+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8381605c0 con 0x7fe83815d3b0 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.727+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe83400f050 con 0x7fe83815d3b0 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.727+0000 7fe82dffb700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe81c0682f0 0x7fe81c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.728+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe8340874a0 con 0x7fe83815d3b0 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.728+0000 7fe83d582700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe81c0682f0 0x7fe81c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.728+0000 7fe83d582700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe81c0682f0 0x7fe81c06a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe82800a8d0 tx=0x7fe828008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.732+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe834053920 con 0x7fe83815d3b0 2023-11-28T11:09:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.929+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8380742b0 con 0x7fe83815d3b0 2023-11-28T11:09:21.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.930+0000 7fe82dffb700 1 -- 172.21.15.83:0/386844112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe834010380 con 0x7fe83815d3b0 2023-11-28T11:09:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe81c0682f0 msgr2=0x7fe81c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe81c0682f0 0x7fe81c06a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe82800a8d0 tx=0x7fe828008040 comp rx=0 tx=0).stop 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 msgr2=0x7fe8380b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 0x7fe8380b3920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe834000c00 tx=0x7fe834004750 comp rx=0 tx=0).stop 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 shutdown_connections 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe838160150 0x7fe8380b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8380cc0f0 0x7fe8380c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe81c0682f0 0x7fe81c06a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 --2- 172.21.15.83:0/386844112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83815d3b0 0x7fe8380b3920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 >> 172.21.15.83:0/386844112 conn(0x7fe8380b1090 msgr2=0x7fe8380cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 shutdown_connections 2023-11-28T11:09:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:21.932+0000 7fe83f7e6700 1 -- 172.21.15.83:0/386844112 wait complete. 2023-11-28T11:09:21.935 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:09:21.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:09:21.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T11:09:21.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:09:21.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.992 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:09:22.009 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:22.009 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:22.031 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:22.031 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:22.091 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:22.091 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:22.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 Processor -- start 2023-11-28T11:09:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 -- start start 2023-11-28T11:09:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 0x7f14100ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f141006cf90 0x7f14100ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1410161450 con 0x7f141015cc20 2023-11-28T11:09:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14100ce630 con 0x7f141006cf90 2023-11-28T11:09:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.271+0000 7f141735a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14101608d0 con 0x7f141015c850 2023-11-28T11:09:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f141006cf90 0x7f14100ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14150f6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 0x7f14100ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f141006cf90 0x7f14100ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:36866/0 (socket says 172.21.15.83:36866) 2023-11-28T11:09:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14150f6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 0x7f14100ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:34070/0 (socket says 172.21.15.83:34070) 2023-11-28T11:09:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14148f5700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f141015cc20 0x7f141006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:46076/0 (socket says 172.21.15.83:46076) 2023-11-28T11:09:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 -- 172.21.15.83:0/1885759982 learned_addr learned my addr 172.21.15.83:0/1885759982 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 -- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 msgr2=0x7f14100ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 --2- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 0x7f14100ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 -- 172.21.15.83:0/1885759982 >> v1:172.21.15.90:6789/0 conn(0x7f141015cc20 legacy=0x7f141006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 -- 172.21.15.83:0/1885759982 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410160bb0 con 0x7f141006cf90 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14150f6700 1 --2- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 0x7f14100ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.272+0000 7f14158f7700 1 --2- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141006cf90 0x7f14100ceff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1408009e50 tx=0x7f140800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6081eb5ee101100d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f1407fff700 1 -- 172.21.15.83:0/1885759982 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800e070 con 0x7f141006cf90 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f1407fff700 1 -- 172.21.15.83:0/1885759982 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140800b910 con 0x7f141006cf90 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f1407fff700 1 -- 172.21.15.83:0/1885759982 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800bcc0 con 0x7f141006cf90 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 -- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141006cf90 msgr2=0x7f14100ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 --2- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141006cf90 0x7f14100ceff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1408009e50 tx=0x7f140800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 -- 172.21.15.83:0/1885759982 shutdown_connections 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 --2- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141006cf90 0x7f14100ceff0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 --2- 172.21.15.83:0/1885759982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141015c850 0x7f14100ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 -- 172.21.15.83:0/1885759982 >> 172.21.15.83:0/1885759982 conn(0x7f14100cd960 msgr2=0x7f14100cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 -- 172.21.15.83:0/1885759982 shutdown_connections 2023-11-28T11:09:22.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.273+0000 7f141735a700 1 -- 172.21.15.83:0/1885759982 wait complete. 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.274+0000 7f141735a700 1 Processor -- start 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.274+0000 7f141735a700 1 -- start start 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f141735a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141006cf90 0x7f141014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f141735a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 0x7f141015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f141735a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141015cc20 0x7f141015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f141735a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141015da40 con 0x7f141015c850 2023-11-28T11:09:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f141735a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141015dd20 con 0x7f141015cc20 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f141735a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141015e000 con 0x7f141006cf90 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f14148f5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 0x7f141015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f14148f5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 0x7f141015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49876/0 (socket says 172.21.15.83:49876) 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.275+0000 7f14148f5700 1 -- 172.21.15.83:0/2235645147 learned_addr learned my addr 172.21.15.83:0/2235645147 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14150f6700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141006cf90 0x7f141014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14158f7700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141015cc20 0x7f141015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14148f5700 1 -- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141006cf90 msgr2=0x7f141014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14148f5700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141006cf90 0x7f141014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14148f5700 1 -- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141015cc20 msgr2=0x7f141015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14148f5700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141015cc20 0x7f141015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14148f5700 1 -- 172.21.15.83:0/2235645147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000094a0 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f14148f5700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 0x7f141015d060 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c4d0 tx=0x7f140c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c005cb0 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c005e10 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1408009b30 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0072d0 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.276+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1410161170 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.277+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1410160e90 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.278+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f140c019030 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.278+0000 7f1405ffb700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13f40682f0 0x7f13f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.278+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f140c0851b0 con 0x7f141015c850 2023-11-28T11:09:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.278+0000 7f14150f6700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13f40682f0 0x7f13f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.278+0000 7f14150f6700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13f40682f0 0x7f13f406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1400009d00 tx=0x7f1400008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.282+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f140c04b5e0 con 0x7f141015c850 2023-11-28T11:09:22.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.464+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1410160560 con 0x7f141015c850 2023-11-28T11:09:22.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.465+0000 7f1405ffb700 1 -- 172.21.15.83:0/2235645147 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f140c017090 con 0x7f141015c850 2023-11-28T11:09:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:22.467 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:08:39.945069+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13f40682f0 msgr2=0x7f13f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13f40682f0 0x7f13f406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1400009d00 tx=0x7f1400008040 comp rx=0 tx=0).stop 2023-11-28T11:09:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 msgr2=0x7f141015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 0x7f141015d060 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c4d0 tx=0x7f140c00c890 comp rx=0 tx=0).stop 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 shutdown_connections 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f141015cc20 0x7f141015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f141006cf90 0x7f141014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13f40682f0 0x7f13f406a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 --2- 172.21.15.83:0/2235645147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015c850 0x7f141015d060 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 >> 172.21.15.83:0/2235645147 conn(0x7f14100cd960 msgr2=0x7f1410158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 shutdown_connections 2023-11-28T11:09:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.467+0000 7f141735a700 1 -- 172.21.15.83:0/2235645147 wait complete. 2023-11-28T11:09:22.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:22.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:22.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.607+0000 7fdc0aff3700 1 Processor -- start 2023-11-28T11:09:22.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.607+0000 7fdc0aff3700 1 -- start start 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.607+0000 7fdc0aff3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdc04156d70 0x7fdc040d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc0aff3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040d0bf0 0x7fdc0415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc0aff3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc0415d1a0 0x7fdc04160250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc0aff3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0415dcc0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc0aff3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc04157140 con 0x7fdc04156d70 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc0aff3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0415d9e0 con 0x7fdc040d0bf0 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc0415d1a0 0x7fdc04160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc0415d1a0 0x7fdc04160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49886/0 (socket says 172.21.15.83:49886) 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 -- 172.21.15.83:0/3937332100 learned_addr learned my addr 172.21.15.83:0/3937332100 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc03fff700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040d0bf0 0x7fdc0415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc08d8f700 1 --2- 172.21.15.83:0/3937332100 >> v2:172.21.15.90:3300/0 conn(0x7fdc04156d70 0x7fdc040d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 -- 172.21.15.83:0/3937332100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040d0bf0 msgr2=0x7fdc0415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040d0bf0 0x7fdc0415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 -- 172.21.15.83:0/3937332100 >> v2:172.21.15.90:3300/0 conn(0x7fdc04156d70 msgr2=0x7fdc040d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 --2- 172.21.15.83:0/3937332100 >> v2:172.21.15.90:3300/0 conn(0x7fdc04156d70 0x7fdc040d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 -- 172.21.15.83:0/3937332100 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0406bdb0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc03fff700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040d0bf0 0x7fdc0415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc09590700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc04160250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc040715e0 tx=0x7fdbfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28072376b936901c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc037fe700 1 -- 172.21.15.83:0/3937332100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00e070 con 0x7fdc0415d1a0 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc037fe700 1 -- 172.21.15.83:0/3937332100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc00b9c0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.608+0000 7fdc037fe700 1 -- 172.21.15.83:0/3937332100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00bd70 con 0x7fdc0415d1a0 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 -- 172.21.15.83:0/3937332100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 msgr2=0x7fdc04160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc04160250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc040715e0 tx=0x7fdbfc00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 -- 172.21.15.83:0/3937332100 shutdown_connections 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc04160250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/3937332100 >> v2:172.21.15.90:3300/0 conn(0x7fdc04156d70 0x7fdc040d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/3937332100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040d0bf0 0x7fdc0415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 -- 172.21.15.83:0/3937332100 >> 172.21.15.83:0/3937332100 conn(0x7fdc0406ea50 msgr2=0x7fdc04070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 -- 172.21.15.83:0/3937332100 shutdown_connections 2023-11-28T11:09:22.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.609+0000 7fdc0aff3700 1 -- 172.21.15.83:0/3937332100 wait complete. 2023-11-28T11:09:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.610+0000 7fdc0aff3700 1 Processor -- start 2023-11-28T11:09:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.610+0000 7fdc0aff3700 1 -- start start 2023-11-28T11:09:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc0aff3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc04156d70 0x7fdc0414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc0aff3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc040c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc0aff3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040c87e0 0x7fdc040b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc0aff3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040b1f00 con 0x7fdc0415d1a0 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc0aff3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0414fd80 con 0x7fdc04156d70 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc0aff3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040b12f0 con 0x7fdc040c87e0 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc040c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc040c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49890/0 (socket says 172.21.15.83:49890) 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 -- 172.21.15.83:0/4213114118 learned_addr learned my addr 172.21.15.83:0/4213114118 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc08d8f700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc04156d70 0x7fdc0414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc09590700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040c87e0 0x7fdc040b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 -- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040c87e0 msgr2=0x7fdc040b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040c87e0 0x7fdc040b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 -- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc04156d70 msgr2=0x7fdc0414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc04156d70 0x7fdc0414f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.611+0000 7fdc03fff700 1 -- 172.21.15.83:0/4213114118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc009b30 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.612+0000 7fdc03fff700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc040c82a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c7f0 tx=0x7fdbf000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.612+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0011800 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.612+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf0011960 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.612+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0406c370 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.612+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0011cf0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.612+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0406c090 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.613+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc040cf1a0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.613+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdbf0021070 con 0x7fdc0415d1a0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.614+0000 7fdc017fa700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdbe80682f0 0x7fdbe806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.614+0000 7fdc08d8f700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdbe80682f0 0x7fdbe806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.614+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdbf0085ca0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.614+0000 7fdc08d8f700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdbe80682f0 0x7fdbe806a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800aae0 tx=0x7fdbf8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.618+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdbf008a070 con 0x7fdc0415d1a0 2023-11-28T11:09:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.797+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc040742b0 con 0x7fdc0415d1a0 2023-11-28T11:09:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.798+0000 7fdc017fa700 1 -- 172.21.15.83:0/4213114118 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdbf001f020 con 0x7fdc0415d1a0 2023-11-28T11:09:22.799 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdbe80682f0 msgr2=0x7fdbe806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdbe80682f0 0x7fdbe806a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800aae0 tx=0x7fdbf8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 msgr2=0x7fdc040c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc040c82a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c7f0 tx=0x7fdbf000cb00 comp rx=0 tx=0).stop 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 shutdown_connections 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc040c87e0 0x7fdc040b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc04156d70 0x7fdc0414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdbe80682f0 0x7fdbe806a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 --2- 172.21.15.83:0/4213114118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc0415d1a0 0x7fdc040c82a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 >> 172.21.15.83:0/4213114118 conn(0x7fdc040d0bf0 msgr2=0x7fdc0406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 shutdown_connections 2023-11-28T11:09:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.800+0000 7fdc0aff3700 1 -- 172.21.15.83:0/4213114118 wait complete. 2023-11-28T11:09:22.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.936+0000 7f925500a700 1 Processor -- start 2023-11-28T11:09:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.936+0000 7f925500a700 1 -- start start 2023-11-28T11:09:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f925500a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f925015c790 0x7f92500b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f925500a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92500b37e0 con 0x7f92500cf040 2023-11-28T11:09:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f925500a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9250160de0 con 0x7f925006d490 2023-11-28T11:09:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f925500a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9250161270 con 0x7f925015c790 2023-11-28T11:09:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f924f59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f92500cf040 0x7f925006bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47760/0 (socket says 172.21.15.83:47760) 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f924f59e700 1 -- 172.21.15.83:0/2763817948 learned_addr learned my addr 172.21.15.83:0/2763817948 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f924ed9d700 1 --2- 172.21.15.83:0/2763817948 >> v2:172.21.15.90:3300/0 conn(0x7f925015c790 0x7f92500b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465908327 0 0) 0x7f9248001490 con 0x7f92500cf040 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.937+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92380035b0 con 0x7f92500cf040 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448974902 0 0) 0x7f92380035b0 con 0x7f92500cf040 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 >> v2:172.21.15.90:3300/0 conn(0x7f925015c790 msgr2=0x7f92500b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 --2- 172.21.15.83:0/2763817948 >> v2:172.21.15.90:3300/0 conn(0x7f925015c790 0x7f92500b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 >> v1:172.21.15.90:6789/0 conn(0x7f925006d490 legacy=0x7f92500ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500b3540 con 0x7f92500cf040 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f9248001b80 con 0x7f92500cf040 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 >> v1:172.21.15.83:6789/0 conn(0x7f92500cf040 legacy=0x7f925006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9238003a00 0x7f9238003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238004850 0x7f9238004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238005200 0x7f923800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9250161270 con 0x7f9238005200 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923800c080 con 0x7f9238004850 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923800c240 con 0x7f9238003a00 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238005200 0x7f923800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924e59c700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238004850 0x7f9238004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924ed9d700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9238003a00 0x7f9238003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 -- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9238003a00 msgr2=0x7f9238003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9238003a00 0x7f9238003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 -- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238004850 msgr2=0x7f9238004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:22.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238004850 0x7f9238004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 -- 172.21.15.83:0/2763817948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380035b0 con 0x7f9238005200 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 -- 172.21.15.83:0/2763817948 reap_dead start 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924f59e700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238005200 0x7f923800ba10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f924800a830 tx=0x7f9238005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d172cfe44e6beb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800a490 con 0x7f9238005200 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248012600 con 0x7f9238005200 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.938+0000 7f924dd9b700 1 -- 172.21.15.83:0/2763817948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248012990 con 0x7f9238005200 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 -- 172.21.15.83:0/2763817948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238005200 msgr2=0x7f923800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238005200 0x7f923800ba10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f924800a830 tx=0x7f9238005880 comp rx=0 tx=0).stop 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 -- 172.21.15.83:0/2763817948 shutdown_connections 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 --2- 172.21.15.83:0/2763817948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238005200 0x7f923800ba10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 -- 172.21.15.83:0/2763817948 >> 172.21.15.83:0/2763817948 conn(0x7f925006f300 msgr2=0x7f925006f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 -- 172.21.15.83:0/2763817948 shutdown_connections 2023-11-28T11:09:22.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.939+0000 7f925500a700 1 -- 172.21.15.83:0/2763817948 wait complete. 2023-11-28T11:09:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.940+0000 7f925500a700 1 Processor -- start 2023-11-28T11:09:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.940+0000 7f925500a700 1 -- start start 2023-11-28T11:09:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f925500a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238005200 0x7f92501470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f925500a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f92500c46c0 0x7f92500b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f925500a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 0x7f92500b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f925500a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92500c5820 con 0x7f92501541d0 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f925500a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92500c4c10 con 0x7f9238005200 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f925500a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92500c4ef0 con 0x7f92500c46c0 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f924f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 0x7f92500b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f924f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 0x7f92500b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49912/0 (socket says 172.21.15.83:49912) 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f924f59e700 1 -- 172.21.15.83:0/1091954804 learned_addr learned my addr 172.21.15.83:0/1091954804 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f924e59c700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f92500c46c0 0x7f92500b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.941+0000 7f924ed9d700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238005200 0x7f92501470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924f59e700 1 -- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f92500c46c0 msgr2=0x7f92500b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924f59e700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f92500c46c0 0x7f92500b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924f59e700 1 -- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238005200 msgr2=0x7f92501470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924f59e700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238005200 0x7f92501470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924f59e700 1 -- 172.21.15.83:0/1091954804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92501604b0 con 0x7f92501541d0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924e59c700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f92500c46c0 0x7f92500b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924ed9d700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238005200 0x7f92501470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f924f59e700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 0x7f92500b1810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f924800a0b0 tx=0x7f9248001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248012480 con 0x7f92501541d0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248001940 con 0x7f92501541d0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925015cb60 con 0x7f92501541d0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924801d420 con 0x7f92501541d0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.942+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9250160790 con 0x7f92501541d0 2023-11-28T11:09:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.943+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925006dc90 con 0x7f92501541d0 2023-11-28T11:09:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.943+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f924802e070 con 0x7f92501541d0 2023-11-28T11:09:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.943+0000 7f92377fe700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f923c0682f0 0x7f923c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.944+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9248087d50 con 0x7f92501541d0 2023-11-28T11:09:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.944+0000 7f924ed9d700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f923c0682f0 0x7f923c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.944+0000 7f924ed9d700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f923c0682f0 0x7f923c06a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9250154180 tx=0x7f9244008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:22.948+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92480542c0 con 0x7f92501541d0 2023-11-28T11:09:23.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.130+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92500b37e0 con 0x7f92501541d0 2023-11-28T11:09:23.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.130+0000 7f92377fe700 1 -- 172.21.15.83:0/1091954804 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92500b37e0 con 0x7f92501541d0 2023-11-28T11:09:23.132 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f923c0682f0 msgr2=0x7f923c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f923c0682f0 0x7f923c06a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9250154180 tx=0x7f9244008040 comp rx=0 tx=0).stop 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 msgr2=0x7f92500b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 0x7f92500b1810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f924800a0b0 tx=0x7f9248001f10 comp rx=0 tx=0).stop 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 shutdown_connections 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f923c0682f0 0x7f923c06a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92501541d0 0x7f92500b1810 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238005200 0x7f92501470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 --2- 172.21.15.83:0/1091954804 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f92500c46c0 0x7f92500b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 >> 172.21.15.83:0/1091954804 conn(0x7f925006f300 msgr2=0x7f9250070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 shutdown_connections 2023-11-28T11:09:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.133+0000 7f925500a700 1 -- 172.21.15.83:0/1091954804 wait complete. 2023-11-28T11:09:23.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.269+0000 7f4f83496700 1 Processor -- start 2023-11-28T11:09:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.269+0000 7f4f83496700 1 -- start start 2023-11-28T11:09:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f83496700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f7c0aee20 0x7f4f7c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f83496700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4f7c06c370 0x7f4f7c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f83496700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c0b5b00 con 0x7f4f7c0af1f0 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f83496700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c06c800 con 0x7f4f7c0aee20 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f83496700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c06cae0 con 0x7f4f7c06c370 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f81232700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f7c0aee20 0x7f4f7c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f81232700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f7c0aee20 0x7f4f7c0b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49916/0 (socket says 172.21.15.83:49916) 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.270+0000 7f4f81232700 1 -- 172.21.15.83:0/2230008137 learned_addr learned my addr 172.21.15.83:0/2230008137 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f80a31700 1 --2- 172.21.15.83:0/2230008137 >> v2:172.21.15.90:3300/0 conn(0x7f4f7c06c370 0x7f4f7c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f81232700 1 -- 172.21.15.83:0/2230008137 >> v2:172.21.15.90:3300/0 conn(0x7f4f7c06c370 msgr2=0x7f4f7c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f81232700 1 --2- 172.21.15.83:0/2230008137 >> v2:172.21.15.90:3300/0 conn(0x7f4f7c06c370 0x7f4f7c06ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f81232700 1 -- 172.21.15.83:0/2230008137 >> v1:172.21.15.83:6789/0 conn(0x7f4f7c0af1f0 legacy=0x7f4f7c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f81232700 1 -- 172.21.15.83:0/2230008137 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c06cdc0 con 0x7f4f7c0aee20 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f80a31700 1 --2- 172.21.15.83:0/2230008137 >> v2:172.21.15.90:3300/0 conn(0x7f4f7c06c370 0x7f4f7c06ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f81232700 1 --2- 172.21.15.83:0/2230008137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c0b55c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0b3bb0 tx=0x7f4f6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3861da68df0d9a7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f73fff700 1 -- 172.21.15.83:0/2230008137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c005e30 con 0x7f4f7c0aee20 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f73fff700 1 -- 172.21.15.83:0/2230008137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c004030 con 0x7f4f7c0aee20 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f73fff700 1 -- 172.21.15.83:0/2230008137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0043e0 con 0x7f4f7c0aee20 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f83496700 1 -- 172.21.15.83:0/2230008137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 msgr2=0x7f4f7c0b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f83496700 1 --2- 172.21.15.83:0/2230008137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c0b55c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0b3bb0 tx=0x7f4f6c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f83496700 1 -- 172.21.15.83:0/2230008137 shutdown_connections 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.271+0000 7f4f83496700 1 --2- 172.21.15.83:0/2230008137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c0b55c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.272+0000 7f4f83496700 1 --2- 172.21.15.83:0/2230008137 >> v2:172.21.15.90:3300/0 conn(0x7f4f7c06c370 0x7f4f7c06ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.272+0000 7f4f83496700 1 -- 172.21.15.83:0/2230008137 >> 172.21.15.83:0/2230008137 conn(0x7f4f7c0b2b60 msgr2=0x7f4f7c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.272+0000 7f4f83496700 1 -- 172.21.15.83:0/2230008137 shutdown_connections 2023-11-28T11:09:23.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.272+0000 7f4f83496700 1 -- 172.21.15.83:0/2230008137 wait complete. 2023-11-28T11:09:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.273+0000 7f4f83496700 1 Processor -- start 2023-11-28T11:09:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.273+0000 7f4f83496700 1 -- start start 2023-11-28T11:09:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.273+0000 7f4f83496700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4f7c06c370 0x7f4f7c0c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.273+0000 7f4f83496700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.273+0000 7f4f83496700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f7c0af1f0 0x7f4f7c1f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f83496700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c15a210 con 0x7f4f7c0aee20 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f83496700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0c5760 con 0x7f4f7c0af1f0 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f83496700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0ae1f0 con 0x7f4f7c06c370 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c1481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49924/0 (socket says 172.21.15.83:49924) 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 -- 172.21.15.83:0/642908451 learned_addr learned my addr 172.21.15.83:0/642908451 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 -- 172.21.15.83:0/642908451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4f7c06c370 msgr2=0x7f4f7c0c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4f7c06c370 0x7f4f7c0c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 -- 172.21.15.83:0/642908451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f7c0af1f0 msgr2=0x7f4f7c1f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f7c0af1f0 0x7f4f7c1f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 -- 172.21.15.83:0/642908451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c009500 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f80a31700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c1481b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c7f0 tx=0x7f4f7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78011800 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f78011960 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c06fc70 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78011cf0 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.274+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c06d0a0 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.275+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7c06d380 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.276+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f78021070 con 0x7f4f7c0aee20 2023-11-28T11:09:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.276+0000 7f4f71ffb700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4f600682f0 0x7f4f6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.276+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4f78085e70 con 0x7f4f7c0aee20 2023-11-28T11:09:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.276+0000 7f4f81232700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4f600682f0 0x7f4f6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.276+0000 7f4f81232700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4f600682f0 0x7f4f6006a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0b3bb0 tx=0x7f4f6c010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.280+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f78052380 con 0x7f4f7c0aee20 2023-11-28T11:09:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.463+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f7c0742b0 con 0x7f4f7c0aee20 2023-11-28T11:09:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.463+0000 7f4f71ffb700 1 -- 172.21.15.83:0/642908451 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f7801f020 con 0x7f4f7c0aee20 2023-11-28T11:09:23.465 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4f600682f0 msgr2=0x7f4f6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4f600682f0 0x7f4f6006a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0b3bb0 tx=0x7f4f6c010000 comp rx=0 tx=0).stop 2023-11-28T11:09:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 msgr2=0x7f4f7c1481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c1481b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c7f0 tx=0x7f4f7800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 shutdown_connections 2023-11-28T11:09:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f7c0af1f0 0x7f4f7c1f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4f7c06c370 0x7f4f7c0c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4f600682f0 0x7f4f6006a7a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 --2- 172.21.15.83:0/642908451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f7c0aee20 0x7f4f7c1481b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 >> 172.21.15.83:0/642908451 conn(0x7f4f7c0b2b60 msgr2=0x7f4f7c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 shutdown_connections 2023-11-28T11:09:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.466+0000 7f4f83496700 1 -- 172.21.15.83:0/642908451 wait complete. 2023-11-28T11:09:23.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:23.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.605+0000 7f6427592700 1 Processor -- start 2023-11-28T11:09:23.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f6427592700 1 -- start start 2023-11-28T11:09:23.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f6427592700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f642006e370 0x7f64200aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f6427592700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 0x7f642006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f6427592700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642006d4a0 con 0x7f642015c890 2023-11-28T11:09:23.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f6427592700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64200b5050 con 0x7f642006e370 2023-11-28T11:09:23.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f6427592700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64200b5ae0 con 0x7f64200b4bb0 2023-11-28T11:09:23.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f642532e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f642015c890 0x7f642006dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47794/0 (socket says 172.21.15.83:47794) 2023-11-28T11:09:23.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.606+0000 7f642532e700 1 -- 172.21.15.83:0/4003276763 learned_addr learned my addr 172.21.15.83:0/4003276763 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:23.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749464409 0 0) 0x7f642006d4a0 con 0x7f642015c890 2023-11-28T11:09:23.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6424b2d700 1 --2- 172.21.15.83:0/4003276763 >> v2:172.21.15.90:3300/0 conn(0x7f642006e370 0x7f64200aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 0x7f642006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f640c0035b0 con 0x7f642015c890 2023-11-28T11:09:23.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660300651 0 0) 0x7f640c0035b0 con 0x7f642015c890 2023-11-28T11:09:23.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 msgr2=0x7f642006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 0x7f642006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 >> v2:172.21.15.90:3300/0 conn(0x7f642006e370 msgr2=0x7f64200aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 --2- 172.21.15.83:0/4003276763 >> v2:172.21.15.90:3300/0 conn(0x7f642006e370 0x7f64200aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642006c400 con 0x7f642015c890 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f641c0017f0 con 0x7f642015c890 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 >> v1:172.21.15.83:6789/0 conn(0x7f642015c890 legacy=0x7f642006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 0x7f640c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c003f80 0x7f640c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f640c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200b5ae0 con 0x7f640c004d90 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c00ba10 con 0x7f640c003f80 2023-11-28T11:09:23.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c00bb80 con 0x7f64200b4bb0 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f640c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f642532e700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 0x7f640c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6424b2d700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c003f80 0x7f640c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 -- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 msgr2=0x7f640c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64200b4bb0 0x7f640c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 -- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c003f80 msgr2=0x7f640c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f640c003f80 0x7f640c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.607+0000 7f6425b2f700 1 -- 172.21.15.83:0/4003276763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0035b0 con 0x7f640c004d90 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6425b2f700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f640c005200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6418009fc0 tx=0x7f640c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8a5657fe2e5b31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641800ebd0 con 0x7f640c004d90 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641800f040 con 0x7f640c004d90 2023-11-28T11:09:23.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6417fff700 1 -- 172.21.15.83:0/4003276763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418014480 con 0x7f640c004d90 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6427592700 1 -- 172.21.15.83:0/4003276763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 msgr2=0x7f640c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6427592700 1 --2- 172.21.15.83:0/4003276763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f640c005200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6418009fc0 tx=0x7f640c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6425b2f700 1 -- 172.21.15.83:0/4003276763 reap_dead start 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6427592700 1 -- 172.21.15.83:0/4003276763 shutdown_connections 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6427592700 1 -- 172.21.15.83:0/4003276763 >> 172.21.15.83:0/4003276763 conn(0x7f64200b2b60 msgr2=0x7f64200b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.608+0000 7f6427592700 1 -- 172.21.15.83:0/4003276763 shutdown_connections 2023-11-28T11:09:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.609+0000 7f6427592700 1 -- 172.21.15.83:0/4003276763 wait complete. 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.610+0000 7f6427592700 1 Processor -- start 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.610+0000 7f6427592700 1 -- start start 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6427592700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f6420155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6427592700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64200c8090 0x7f64200ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6427592700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f642015a110 0x7f642015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6427592700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642015aaf0 con 0x7f640c004d90 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6427592700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200c9130 con 0x7f64200c8090 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6427592700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64201555d0 con 0x7f642015a110 2023-11-28T11:09:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f6420155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f6420155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49946/0 (socket says 172.21.15.83:49946) 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 -- 172.21.15.83:0/2786708132 learned_addr learned my addr 172.21.15.83:0/2786708132 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6424b2d700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64200c8090 0x7f64200ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f6425b2f700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f642015a110 0x7f642015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 -- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f642015a110 msgr2=0x7f642015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f642015a110 0x7f642015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 -- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64200c8090 msgr2=0x7f64200ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64200c8090 0x7f64200ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.611+0000 7f642532e700 1 -- 172.21.15.83:0/2786708132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64200b5520 con 0x7f640c004d90 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.612+0000 7f642532e700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f6420155090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f641c007d70 tx=0x7f641c001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.612+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0072e0 con 0x7f640c004d90 2023-11-28T11:09:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.612+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c007440 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.612+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64200b5240 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.612+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c005830 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.612+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64200b5800 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.613+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64200b3820 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.613+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f641c005990 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.614+0000 7f6415ffb700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64040682f0 0x7f640406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.614+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f641c086240 con 0x7f640c004d90 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.614+0000 7f6424b2d700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64040682f0 0x7f640406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.614+0000 7f6424b2d700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64040682f0 0x7f640406a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64200ade00 tx=0x7f6410008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.618+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f641c0527b0 con 0x7f640c004d90 2023-11-28T11:09:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.799+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64200742b0 con 0x7f640c004d90 2023-11-28T11:09:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.799+0000 7f6415ffb700 1 -- 172.21.15.83:0/2786708132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f641c055dd0 con 0x7f640c004d90 2023-11-28T11:09:23.801 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64040682f0 msgr2=0x7f640406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64040682f0 0x7f640406a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64200ade00 tx=0x7f6410008040 comp rx=0 tx=0).stop 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 msgr2=0x7f6420155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f6420155090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f641c007d70 tx=0x7f641c001890 comp rx=0 tx=0).stop 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 shutdown_connections 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f642015a110 0x7f642015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64040682f0 0x7f640406a7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c004d90 0x7f6420155090 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 --2- 172.21.15.83:0/2786708132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64200c8090 0x7f64200ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 >> 172.21.15.83:0/2786708132 conn(0x7f64200b2b60 msgr2=0x7f64200ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 shutdown_connections 2023-11-28T11:09:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.801+0000 7f6427592700 1 -- 172.21.15.83:0/2786708132 wait complete. 2023-11-28T11:09:23.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 Processor -- start 2023-11-28T11:09:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 -- start start 2023-11-28T11:09:23.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ea8158c50 0x7f7ea80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea80c9620 0x7f7ea8071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7ea80718a0 0x7f7ea806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80cab20 con 0x7f7ea8158c50 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea8159020 con 0x7f7ea80718a0 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.937+0000 7f7ead338700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8071d20 con 0x7f7ea80c9620 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ea8158c50 0x7f7ea80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ea8158c50 0x7f7ea80cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49954/0 (socket says 172.21.15.83:49954) 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 -- 172.21.15.83:0/1095207720 learned_addr learned my addr 172.21.15.83:0/1095207720 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea67fc700 1 --2- 172.21.15.83:0/1095207720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea80c9620 0x7f7ea8071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea77fe700 1 --2- 172.21.15.83:0/1095207720 >> v2:172.21.15.90:3300/0 conn(0x7f7ea80718a0 0x7f7ea806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 -- 172.21.15.83:0/1095207720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea80c9620 msgr2=0x7f7ea8071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 --2- 172.21.15.83:0/1095207720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea80c9620 0x7f7ea8071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 -- 172.21.15.83:0/1095207720 >> v2:172.21.15.90:3300/0 conn(0x7f7ea80718a0 msgr2=0x7f7ea806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 --2- 172.21.15.83:0/1095207720 >> v2:172.21.15.90:3300/0 conn(0x7f7ea80718a0 0x7f7ea806a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 -- 172.21.15.83:0/1095207720 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80c9ea0 con 0x7f7ea8158c50 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea77fe700 1 --2- 172.21.15.83:0/1095207720 >> v2:172.21.15.90:3300/0 conn(0x7f7ea80718a0 0x7f7ea806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea6ffd700 1 --2- 172.21.15.83:0/1095207720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8158c50 0x7f7ea80cd670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ea806b540 tx=0x7f7e98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714e394dc7f13704 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea5ffb700 1 -- 172.21.15.83:0/1095207720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011830 con 0x7f7ea8158c50 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea5ffb700 1 -- 172.21.15.83:0/1095207720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98011990 con 0x7f7ea8158c50 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.938+0000 7f7ea5ffb700 1 -- 172.21.15.83:0/1095207720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011d20 con 0x7f7ea8158c50 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 -- 172.21.15.83:0/1095207720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8158c50 msgr2=0x7f7ea80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 --2- 172.21.15.83:0/1095207720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8158c50 0x7f7ea80cd670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ea806b540 tx=0x7f7e98009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 -- 172.21.15.83:0/1095207720 shutdown_connections 2023-11-28T11:09:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 --2- 172.21.15.83:0/1095207720 >> v2:172.21.15.90:3300/0 conn(0x7f7ea80718a0 0x7f7ea806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 --2- 172.21.15.83:0/1095207720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8158c50 0x7f7ea80cd670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 --2- 172.21.15.83:0/1095207720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea80c9620 0x7f7ea8071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 -- 172.21.15.83:0/1095207720 >> 172.21.15.83:0/1095207720 conn(0x7f7ea80ceb80 msgr2=0x7f7ea80cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 -- 172.21.15.83:0/1095207720 shutdown_connections 2023-11-28T11:09:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.939+0000 7f7ead338700 1 -- 172.21.15.83:0/1095207720 wait complete. 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.940+0000 7f7ead338700 1 Processor -- start 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.940+0000 7f7ead338700 1 -- start start 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ead338700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea80718a0 0x7f7ea80c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ead338700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 0x7f7ea8140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ead338700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea8158c50 0x7f7ea8155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ead338700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8155070 con 0x7f7ea80c9620 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ead338700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81543f0 con 0x7f7ea80718a0 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ead338700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81546d0 con 0x7f7ea8158c50 2023-11-28T11:09:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 0x7f7ea8140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 0x7f7ea8140420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49962/0 (socket says 172.21.15.83:49962) 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 -- 172.21.15.83:0/1249782698 learned_addr learned my addr 172.21.15.83:0/1249782698 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea6ffd700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea80718a0 0x7f7ea80c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea77fe700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea8158c50 0x7f7ea8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 -- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea8158c50 msgr2=0x7f7ea8155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea8158c50 0x7f7ea8155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 -- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea80718a0 msgr2=0x7f7ea80c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea80718a0 0x7f7ea80c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 -- 172.21.15.83:0/1249782698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009550 con 0x7f7ea80c9620 2023-11-28T11:09:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.941+0000 7f7ea67fc700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 0x7f7ea8140420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e9b0 tx=0x7f7e9c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.942+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c009710 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.942+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00ee50 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.942+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea80ca460 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.942+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00ca50 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.942+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea80ca180 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.943+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea80ca740 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.943+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e9c019030 con 0x7f7ea80c9620 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.943+0000 7f7e8f7fe700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7e940682f0 0x7f7e9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.944+0000 7f7ea6ffd700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7e940682f0 0x7f7e9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.944+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7e9c085e80 con 0x7f7ea80c9620 2023-11-28T11:09:23.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.944+0000 7f7ea6ffd700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7e940682f0 0x7f7e9406a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e98014040 tx=0x7f7e98008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:23.948+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7e9c052390 con 0x7f7ea80c9620 2023-11-28T11:09:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.131+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7ea81551b0 con 0x7f7ea80c9620 2023-11-28T11:09:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.132+0000 7f7e8f7fe700 1 -- 172.21.15.83:0/1249782698 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ea81551b0 con 0x7f7ea80c9620 2023-11-28T11:09:24.133 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7e940682f0 msgr2=0x7f7e9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7e940682f0 0x7f7e9406a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e98014040 tx=0x7f7e98008030 comp rx=0 tx=0).stop 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 msgr2=0x7f7ea8140420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 0x7f7ea8140420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e9b0 tx=0x7f7e9c00ecc0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 shutdown_connections 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ea8158c50 0x7f7ea8155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea80718a0 0x7f7ea80c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7e940682f0 0x7f7e9406a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 --2- 172.21.15.83:0/1249782698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80c9620 0x7f7ea8140420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 >> 172.21.15.83:0/1249782698 conn(0x7f7ea80ceb80 msgr2=0x7f7ea8159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 shutdown_connections 2023-11-28T11:09:24.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.134+0000 7f7ead338700 1 -- 172.21.15.83:0/1249782698 wait complete. 2023-11-28T11:09:24.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.268+0000 7f7a8e947700 1 Processor -- start 2023-11-28T11:09:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.268+0000 7f7a8e947700 1 -- start start 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8e947700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7a880cc0f0 0x7f7a880cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8e947700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a8815d3b0 0x7f7a880ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8e947700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a88160930 con 0x7f7a88160150 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8e947700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a88160c10 con 0x7f7a8815d3b0 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8e947700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a88160ef0 con 0x7f7a880cc0f0 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a877fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7a88160150 0x7f7a880cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47828/0 (socket says 172.21.15.83:47828) 2023-11-28T11:09:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a877fe700 1 -- 172.21.15.83:0/3860235024 learned_addr learned my addr 172.21.15.83:0/3860235024 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8cee4700 1 --2- 172.21.15.83:0/3860235024 >> v2:172.21.15.83:3300/0 conn(0x7f7a8815d3b0 0x7f7a880ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a87fff700 1 --2- 172.21.15.83:0/3860235024 >> v2:172.21.15.90:3300/0 conn(0x7f7a880cc0f0 0x7f7a880cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8cee4700 1 -- 172.21.15.83:0/3860235024 >> v2:172.21.15.90:3300/0 conn(0x7f7a880cc0f0 msgr2=0x7f7a880cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8cee4700 1 --2- 172.21.15.83:0/3860235024 >> v2:172.21.15.90:3300/0 conn(0x7f7a880cc0f0 0x7f7a880cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8cee4700 1 -- 172.21.15.83:0/3860235024 >> v1:172.21.15.83:6789/0 conn(0x7f7a88160150 legacy=0x7f7a880cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.269+0000 7f7a8cee4700 1 -- 172.21.15.83:0/3860235024 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8815ce90 con 0x7f7a8815d3b0 2023-11-28T11:09:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8cee4700 1 --2- 172.21.15.83:0/3860235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880ccf30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a88070750 tx=0x7f7a8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94555f02104de046 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a86ffd700 1 -- 172.21.15.83:0/3860235024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000e070 con 0x7f7a8815d3b0 2023-11-28T11:09:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a86ffd700 1 -- 172.21.15.83:0/3860235024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8000bbd0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a86ffd700 1 -- 172.21.15.83:0/3860235024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80012430 con 0x7f7a8815d3b0 2023-11-28T11:09:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 -- 172.21.15.83:0/3860235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 msgr2=0x7f7a880ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3860235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880ccf30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a88070750 tx=0x7f7a8000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 -- 172.21.15.83:0/3860235024 shutdown_connections 2023-11-28T11:09:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3860235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880ccf30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3860235024 >> v2:172.21.15.90:3300/0 conn(0x7f7a880cc0f0 0x7f7a880cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 -- 172.21.15.83:0/3860235024 >> 172.21.15.83:0/3860235024 conn(0x7f7a880b1090 msgr2=0x7f7a880b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 -- 172.21.15.83:0/3860235024 shutdown_connections 2023-11-28T11:09:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.270+0000 7f7a8e947700 1 -- 172.21.15.83:0/3860235024 wait complete. 2023-11-28T11:09:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.271+0000 7f7a8e947700 1 Processor -- start 2023-11-28T11:09:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 -- start start 2023-11-28T11:09:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a880cc0f0 0x7f7a881541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a88160150 0x7f7a880c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881e6df0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88154a70 con 0x7f7a880cc0f0 2023-11-28T11:09:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a8e947700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881e5f00 con 0x7f7a88160150 2023-11-28T11:09:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.272+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49974/0 (socket says 172.21.15.83:49974) 2023-11-28T11:09:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 -- 172.21.15.83:0/3750393960 learned_addr learned my addr 172.21.15.83:0/3750393960 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a87fff700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a880cc0f0 0x7f7a881541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 -- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a88160150 msgr2=0x7f7a880c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a88160150 0x7f7a880c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 -- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a880cc0f0 msgr2=0x7f7a881541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a880cc0f0 0x7f7a881541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 -- 172.21.15.83:0/3750393960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80009a70 con 0x7f7a8815d3b0 2023-11-28T11:09:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a877fe700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880b3b20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c480 tx=0x7f7a7800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78005d70 con 0x7f7a8815d3b0 2023-11-28T11:09:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780070d0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a881605f0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78007440 con 0x7f7a8815d3b0 2023-11-28T11:09:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.273+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8815cb60 con 0x7f7a8815d3b0 2023-11-28T11:09:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.274+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a880ca330 con 0x7f7a8815d3b0 2023-11-28T11:09:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.275+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a78019030 con 0x7f7a8815d3b0 2023-11-28T11:09:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.275+0000 7f7a84ff9700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a740682f0 0x7f7a7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.275+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7a780852a0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.275+0000 7f7a87fff700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a740682f0 0x7f7a7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.275+0000 7f7a87fff700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a740682f0 0x7f7a7406a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009950 tx=0x7f7a7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.279+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a7804b5e0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.478+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a880742b0 con 0x7f7a8815d3b0 2023-11-28T11:09:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.478+0000 7f7a84ff9700 1 -- 172.21.15.83:0/3750393960 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7a78051880 con 0x7f7a8815d3b0 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a740682f0 msgr2=0x7f7a7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a740682f0 0x7f7a7406a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009950 tx=0x7f7a7c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 msgr2=0x7f7a880b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880b3b20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c480 tx=0x7f7a7800c840 comp rx=0 tx=0).stop 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 shutdown_connections 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7a88160150 0x7f7a880c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a880cc0f0 0x7f7a881541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7a740682f0 0x7f7a7406a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 --2- 172.21.15.83:0/3750393960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8815d3b0 0x7f7a880b3b20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.480+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 >> 172.21.15.83:0/3750393960 conn(0x7f7a880b1090 msgr2=0x7f7a88155960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.481+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 shutdown_connections 2023-11-28T11:09:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.481+0000 7f7a8e947700 1 -- 172.21.15.83:0/3750393960 wait complete. 2023-11-28T11:09:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:09:24.494 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T11:09:24.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:24.494 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T11:09:24.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:24.535 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:09:24.603 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:24.603 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:24.620 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:24.621 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:24.678 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:24.678 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:24.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:24.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.894+0000 7f8e43153700 1 Processor -- start 2023-11-28T11:09:24.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.894+0000 7f8e43153700 1 -- start start 2023-11-28T11:09:24.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e43153700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e3c0cf0d0 0x7f8e3c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e43153700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0cf540 con 0x7f8e3c159830 2023-11-28T11:09:24.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e43153700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0cf790 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e43153700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c15d480 con 0x7f8e3c0cf0d0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e3bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e3c0cf0d0 0x7f8e3c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e3bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e3c0cf0d0 0x7f8e3c06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49982/0 (socket says 172.21.15.83:49982) 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e3bfff700 1 -- 172.21.15.83:0/3734617205 learned_addr learned my addr 172.21.15.83:0/3734617205 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e3bfff700 1 -- 172.21.15.83:0/3734617205 >> v1:172.21.15.90:6789/0 conn(0x7f8e3c0b18a0 legacy=0x7f8e3c0cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.895+0000 7f8e3bfff700 1 -- 172.21.15.83:0/3734617205 >> v1:172.21.15.83:6789/0 conn(0x7f8e3c159830 legacy=0x7f8e3c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e3bfff700 1 -- 172.21.15.83:0/3734617205 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0cfa70 con 0x7f8e3c0cf0d0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e3bfff700 1 --2- 172.21.15.83:0/3734617205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0cf0d0 0x7f8e3c06c5f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8e280098f0 tx=0x7f8e28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf9f7287fbd96be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e3b7fe700 1 -- 172.21.15.83:0/3734617205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28004230 con 0x7f8e3c0cf0d0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e3b7fe700 1 -- 172.21.15.83:0/3734617205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28004390 con 0x7f8e3c0cf0d0 2023-11-28T11:09:24.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e3b7fe700 1 -- 172.21.15.83:0/3734617205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e280036a0 con 0x7f8e3c0cf0d0 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 -- 172.21.15.83:0/3734617205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0cf0d0 msgr2=0x7f8e3c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 --2- 172.21.15.83:0/3734617205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0cf0d0 0x7f8e3c06c5f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8e280098f0 tx=0x7f8e28009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 -- 172.21.15.83:0/3734617205 shutdown_connections 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 --2- 172.21.15.83:0/3734617205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0cf0d0 0x7f8e3c06c5f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 -- 172.21.15.83:0/3734617205 >> 172.21.15.83:0/3734617205 conn(0x7f8e3c06e9a0 msgr2=0x7f8e3c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 -- 172.21.15.83:0/3734617205 shutdown_connections 2023-11-28T11:09:24.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.896+0000 7f8e43153700 1 -- 172.21.15.83:0/3734617205 wait complete. 2023-11-28T11:09:24.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.898+0000 7f8e43153700 1 Processor -- start 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.898+0000 7f8e43153700 1 -- start start 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.898+0000 7f8e43153700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 0x7f8e3c1f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.898+0000 7f8e43153700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e3c0cf0d0 0x7f8e3c14b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.898+0000 7f8e43153700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3c159830 0x7f8e3c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.898+0000 7f8e43153700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0c4d10 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e43153700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c15b6d0 con 0x7f8e3c159830 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e43153700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c15b9b0 con 0x7f8e3c0cf0d0 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 0x7f8e3c1f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 0x7f8e3c1f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49986/0 (socket says 172.21.15.83:49986) 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 -- 172.21.15.83:0/316712860 learned_addr learned my addr 172.21.15.83:0/316712860 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:24.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e416f0700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3c159830 0x7f8e3c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e3bfff700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e3c0cf0d0 0x7f8e3c14b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 -- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e3c0cf0d0 msgr2=0x7f8e3c14b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e3c0cf0d0 0x7f8e3c14b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 -- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3c159830 msgr2=0x7f8e3c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3c159830 0x7f8e3c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 -- 172.21.15.83:0/316712860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28009550 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e40eef700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 0x7f8e3c1f0360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c430 tx=0x7f8e3000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e300050d0 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30005250 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c15d970 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30006860 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.899+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c15d690 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.900+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c159c00 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.901+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e30010050 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.901+0000 7f8e397fa700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e200682f0 0x7f8e2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.901+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8e30084f60 con 0x7f8e3c0b18a0 2023-11-28T11:09:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.901+0000 7f8e3bfff700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e200682f0 0x7f8e2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.902+0000 7f8e3bfff700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e200682f0 0x7f8e2006a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009130 tx=0x7f8e28008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:24.905+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8e3004a5e0 con 0x7f8e3c0b18a0 2023-11-28T11:09:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.097+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e3c074320 con 0x7f8e3c0b18a0 2023-11-28T11:09:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.099+0000 7f8e397fa700 1 -- 172.21.15.83:0/316712860 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f8e30051490 con 0x7f8e3c0b18a0 2023-11-28T11:09:25.100 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:25.100 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:25.004035+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.100+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e200682f0 msgr2=0x7f8e2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.100+0000 7f8e43153700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e200682f0 0x7f8e2006a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009130 tx=0x7f8e28008030 comp rx=0 tx=0).stop 2023-11-28T11:09:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.100+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 msgr2=0x7f8e3c1f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.100+0000 7f8e43153700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 0x7f8e3c1f0360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c430 tx=0x7f8e3000c740 comp rx=0 tx=0).stop 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 shutdown_connections 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3c159830 0x7f8e3c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e200682f0 0x7f8e2006a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e3c0b18a0 0x7f8e3c1f0360 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 --2- 172.21.15.83:0/316712860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e3c0cf0d0 0x7f8e3c14b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 >> 172.21.15.83:0/316712860 conn(0x7f8e3c06e9a0 msgr2=0x7f8e3c154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 shutdown_connections 2023-11-28T11:09:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.101+0000 7f8e43153700 1 -- 172.21.15.83:0/316712860 wait complete. 2023-11-28T11:09:25.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:25.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.236+0000 7fdb02764700 1 Processor -- start 2023-11-28T11:09:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.236+0000 7fdb02764700 1 -- start start 2023-11-28T11:09:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.236+0000 7fdb02764700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdafc0aee50 0x7fdafc0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.236+0000 7fdb02764700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdafc0af220 0x7fdafc15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdb02764700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdafc0d0970 con 0x7fdafc15d8b0 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdb02764700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdafc15dcf0 con 0x7fdafc0aee50 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdb02764700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdafc15dee0 con 0x7fdafc0af220 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdafc0aee50 0x7fdafc0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdafc0aee50 0x7fdafc0d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49994/0 (socket says 172.21.15.83:49994) 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 -- 172.21.15.83:0/3658013187 learned_addr learned my addr 172.21.15.83:0/3658013187 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdaf3fff700 1 --2- 172.21.15.83:0/3658013187 >> v2:172.21.15.90:3300/0 conn(0x7fdafc0af220 0x7fdafc15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 -- 172.21.15.83:0/3658013187 >> v2:172.21.15.90:3300/0 conn(0x7fdafc0af220 msgr2=0x7fdafc15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 --2- 172.21.15.83:0/3658013187 >> v2:172.21.15.90:3300/0 conn(0x7fdafc0af220 0x7fdafc15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 -- 172.21.15.83:0/3658013187 >> v1:172.21.15.90:6789/0 conn(0x7fdafc15d8b0 legacy=0x7fdafc06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 -- 172.21.15.83:0/3658013187 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc0cf950 con 0x7fdafc0aee50 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafbfff700 1 --2- 172.21.15.83:0/3658013187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc0aee50 0x7fdafc0d0430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdafc070b50 tx=0x7fdaec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eb4f36558fdb7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafb7fe700 1 -- 172.21.15.83:0/3658013187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00b870 con 0x7fdafc0aee50 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafb7fe700 1 -- 172.21.15.83:0/3658013187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec00b9d0 con 0x7fdafc0aee50 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.237+0000 7fdafb7fe700 1 -- 172.21.15.83:0/3658013187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00bd60 con 0x7fdafc0aee50 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 -- 172.21.15.83:0/3658013187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc0aee50 msgr2=0x7fdafc0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 --2- 172.21.15.83:0/3658013187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc0aee50 0x7fdafc0d0430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdafc070b50 tx=0x7fdaec009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 -- 172.21.15.83:0/3658013187 shutdown_connections 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 --2- 172.21.15.83:0/3658013187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc0aee50 0x7fdafc0d0430 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 --2- 172.21.15.83:0/3658013187 >> v2:172.21.15.90:3300/0 conn(0x7fdafc0af220 0x7fdafc15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 -- 172.21.15.83:0/3658013187 >> 172.21.15.83:0/3658013187 conn(0x7fdafc0b2b30 msgr2=0x7fdafc0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 -- 172.21.15.83:0/3658013187 shutdown_connections 2023-11-28T11:09:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.238+0000 7fdb02764700 1 -- 172.21.15.83:0/3658013187 wait complete. 2023-11-28T11:09:25.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.239+0000 7fdb02764700 1 Processor -- start 2023-11-28T11:09:25.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.239+0000 7fdb02764700 1 -- start start 2023-11-28T11:09:25.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb02764700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdafc0aee50 0x7fdafc0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb02764700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdafc0af220 0x7fdafc0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb02764700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 0x7fdafc0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb02764700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc153960 con 0x7fdafc15d8b0 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb02764700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc0c45e0 con 0x7fdafc0aee50 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb02764700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc0c5b60 con 0x7fdafc0af220 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 0x7fdafc0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 0x7fdafc0c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50004/0 (socket says 172.21.15.83:50004) 2023-11-28T11:09:25.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 -- 172.21.15.83:0/3128825817 learned_addr learned my addr 172.21.15.83:0/3128825817 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdafbfff700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdafc0aee50 0x7fdafc0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdaf3fff700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdafc0af220 0x7fdafc0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 -- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdafc0af220 msgr2=0x7fdafc0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdafc0af220 0x7fdafc0c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 -- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdafc0aee50 msgr2=0x7fdafc0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdafc0aee50 0x7fdafc0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 -- 172.21.15.83:0/3128825817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec009550 con 0x7fdafc15d8b0 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdb00d01700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 0x7fdafc0c6170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400cc50 tx=0x7fdaf400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.240+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4003c00 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.241+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf4003d60 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.241+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf400aab0 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.241+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc0cff10 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.241+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc0cfc30 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.241+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc0b1910 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.242+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdaf4010050 con 0x7fdafc15d8b0 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.242+0000 7fdaf97fa700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdadc0682f0 0x7fdadc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.242+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdaf4085df0 con 0x7fdafc15d8b0 2023-11-28T11:09:25.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.242+0000 7fdafbfff700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdadc0682f0 0x7fdadc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.243+0000 7fdafbfff700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdadc0682f0 0x7fdadc06a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdaec014040 tx=0x7fdaec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.247+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdaf4052320 con 0x7fdafc15d8b0 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.426+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdafc0742b0 con 0x7fdafc15d8b0 2023-11-28T11:09:25.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.426+0000 7fdaf97fa700 1 -- 172.21.15.83:0/3128825817 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdaf4055940 con 0x7fdafc15d8b0 2023-11-28T11:09:25.428 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdadc0682f0 msgr2=0x7fdadc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdadc0682f0 0x7fdadc06a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdaec014040 tx=0x7fdaec008030 comp rx=0 tx=0).stop 2023-11-28T11:09:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 msgr2=0x7fdafc0c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 0x7fdafc0c6170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400cc50 tx=0x7fdaf400cf60 comp rx=0 tx=0).stop 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 shutdown_connections 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdadc0682f0 0x7fdadc06a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc15d8b0 0x7fdafc0c6170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.428+0000 7fdb02764700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdafc0aee50 0x7fdafc0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.429+0000 7fdb02764700 1 --2- 172.21.15.83:0/3128825817 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdafc0af220 0x7fdafc0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.429+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 >> 172.21.15.83:0/3128825817 conn(0x7fdafc0b2b30 msgr2=0x7fdafc0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.429+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 shutdown_connections 2023-11-28T11:09:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.429+0000 7fdb02764700 1 -- 172.21.15.83:0/3128825817 wait complete. 2023-11-28T11:09:25.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.563+0000 7fedd7ae4700 1 Processor -- start 2023-11-28T11:09:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.563+0000 7fedd7ae4700 1 -- start start 2023-11-28T11:09:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd7ae4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedd006e370 0x7fedd00aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd7ae4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd00b4bb0 0x7fedd006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd7ae4700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd006d4a0 con 0x7fedd015c890 2023-11-28T11:09:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd7ae4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00b5050 con 0x7fedd006e370 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd7ae4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00b5ae0 con 0x7fedd00b4bb0 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd507f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedd006e370 0x7fedd00aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd507f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedd006e370 0x7fedd00aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50010/0 (socket says 172.21.15.83:50010) 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.564+0000 7fedd507f700 1 -- 172.21.15.83:0/787592453 learned_addr learned my addr 172.21.15.83:0/787592453 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd6081700 1 --2- 172.21.15.83:0/787592453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd00b4bb0 0x7fedd006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd507f700 1 -- 172.21.15.83:0/787592453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd00b4bb0 msgr2=0x7fedd006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd507f700 1 --2- 172.21.15.83:0/787592453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd00b4bb0 0x7fedd006b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd507f700 1 -- 172.21.15.83:0/787592453 >> v1:172.21.15.90:6789/0 conn(0x7fedd015c890 legacy=0x7fedd006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd507f700 1 -- 172.21.15.83:0/787592453 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd006c400 con 0x7fedd006e370 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd507f700 1 --2- 172.21.15.83:0/787592453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd006e370 0x7fedd00aee60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0098f0 tx=0x7fedcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb1a4c8b2acdc30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd487e700 1 -- 172.21.15.83:0/787592453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc004230 con 0x7fedd006e370 2023-11-28T11:09:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd487e700 1 -- 172.21.15.83:0/787592453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc004390 con 0x7fedd006e370 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd487e700 1 -- 172.21.15.83:0/787592453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0036a0 con 0x7fedd006e370 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd7ae4700 1 -- 172.21.15.83:0/787592453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd006e370 msgr2=0x7fedd00aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.565+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/787592453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd006e370 0x7fedd00aee60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0098f0 tx=0x7fedcc009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.566+0000 7fedd7ae4700 1 -- 172.21.15.83:0/787592453 shutdown_connections 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.566+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/787592453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd00b4bb0 0x7fedd006b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.566+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/787592453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd006e370 0x7fedd00aee60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.566+0000 7fedd7ae4700 1 -- 172.21.15.83:0/787592453 >> 172.21.15.83:0/787592453 conn(0x7fedd00b2b60 msgr2=0x7fedd00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.566+0000 7fedd7ae4700 1 -- 172.21.15.83:0/787592453 shutdown_connections 2023-11-28T11:09:25.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.566+0000 7fedd7ae4700 1 -- 172.21.15.83:0/787592453 wait complete. 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.567+0000 7fedd7ae4700 1 Processor -- start 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.567+0000 7fedd7ae4700 1 -- start start 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd006e370 0x7fedd0148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 0x7fedd0147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd015c890 0x7fedd00c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd01e86a0 con 0x7fedd00b4bb0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c57a0 con 0x7fedd015c890 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c4ee0 con 0x7fedd006e370 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 0x7fedd0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 0x7fedd0147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50016/0 (socket says 172.21.15.83:50016) 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 -- 172.21.15.83:0/3991563281 learned_addr learned my addr 172.21.15.83:0/3991563281 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 -- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd006e370 msgr2=0x7fedd0148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd006e370 0x7fedd0148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 -- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd015c890 msgr2=0x7fedd00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd6081700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd015c890 0x7fedd00c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd015c890 0x7fedd00c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 -- 172.21.15.83:0/3991563281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc009550 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd507f700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 0x7fedd0147740 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fedd008a420 tx=0x7fedcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00b860 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc00b9e0 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd00b5520 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00bd90 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.568+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd00b5240 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.569+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd00b5800 con 0x7fedd00b4bb0 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.570+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedcc017020 con 0x7fedd00b4bb0 2023-11-28T11:09:25.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.570+0000 7fedc67fc700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb40682f0 0x7fedb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.570+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fedcc0877e0 con 0x7fedd00b4bb0 2023-11-28T11:09:25.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.570+0000 7fedd5880700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb40682f0 0x7fedb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.570+0000 7fedd5880700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb40682f0 0x7fedb406a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedc00096f0 tx=0x7fedc0006c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.574+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedcc053d10 con 0x7fedd00b4bb0 2023-11-28T11:09:25.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.756+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedd00c5270 con 0x7fedd00b4bb0 2023-11-28T11:09:25.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.757+0000 7fedc67fc700 1 -- 172.21.15.83:0/3991563281 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fedcc057330 con 0x7fedd00b4bb0 2023-11-28T11:09:25.758 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:25.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb40682f0 msgr2=0x7fedb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb40682f0 0x7fedb406a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedc00096f0 tx=0x7fedc0006c40 comp rx=0 tx=0).stop 2023-11-28T11:09:25.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 msgr2=0x7fedd0147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 0x7fedd0147740 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fedd008a420 tx=0x7fedcc004750 comp rx=0 tx=0).stop 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 shutdown_connections 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd015c890 0x7fedd00c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fedd006e370 0x7fedd0148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fedb40682f0 0x7fedb406a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 --2- 172.21.15.83:0/3991563281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd00b4bb0 0x7fedd0147740 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 >> 172.21.15.83:0/3991563281 conn(0x7fedd00b2b60 msgr2=0x7fedd00ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 shutdown_connections 2023-11-28T11:09:25.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.759+0000 7fedd7ae4700 1 -- 172.21.15.83:0/3991563281 wait complete. 2023-11-28T11:09:25.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.893+0000 7f31ef5e3700 1 Processor -- start 2023-11-28T11:09:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.894+0000 7f31ef5e3700 1 -- start start 2023-11-28T11:09:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.894+0000 7f31ef5e3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31e815da70 0x7f31e815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.894+0000 7f31ef5e3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f31e80ca060 0x7f31e81604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.894+0000 7f31ef5e3700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80cd560 con 0x7f31e80cced0 2023-11-28T11:09:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.894+0000 7f31ef5e3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e80cd820 con 0x7f31e815da70 2023-11-28T11:09:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.894+0000 7f31ef5e3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e80cdb00 con 0x7f31e80ca060 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31e815da70 0x7f31e815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31e815da70 0x7f31e815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50022/0 (socket says 172.21.15.83:50022) 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 -- 172.21.15.83:0/1854938466 learned_addr learned my addr 172.21.15.83:0/1854938466 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31edb80700 1 --2- 172.21.15.83:0/1854938466 >> v2:172.21.15.90:3300/0 conn(0x7f31e80ca060 0x7f31e81604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 -- 172.21.15.83:0/1854938466 >> v2:172.21.15.90:3300/0 conn(0x7f31e80ca060 msgr2=0x7f31e81604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 --2- 172.21.15.83:0/1854938466 >> v2:172.21.15.90:3300/0 conn(0x7f31e80ca060 0x7f31e81604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 -- 172.21.15.83:0/1854938466 >> v1:172.21.15.90:6789/0 conn(0x7f31e80cced0 legacy=0x7f31e80c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 -- 172.21.15.83:0/1854938466 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e80cdde0 con 0x7f31e815da70 2023-11-28T11:09:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31ed37f700 1 --2- 172.21.15.83:0/1854938466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e815da70 0x7f31e815cd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f31e80aed20 tx=0x7f31d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b293495bfd200cf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31dffff700 1 -- 172.21.15.83:0/1854938466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800b870 con 0x7f31e815da70 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31dffff700 1 -- 172.21.15.83:0/1854938466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d800b9d0 con 0x7f31e815da70 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.895+0000 7f31dffff700 1 -- 172.21.15.83:0/1854938466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800bd60 con 0x7f31e815da70 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 -- 172.21.15.83:0/1854938466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e815da70 msgr2=0x7f31e815cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/1854938466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e815da70 0x7f31e815cd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f31e80aed20 tx=0x7f31d8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 -- 172.21.15.83:0/1854938466 shutdown_connections 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/1854938466 >> v2:172.21.15.90:3300/0 conn(0x7f31e80ca060 0x7f31e81604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/1854938466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e815da70 0x7f31e815cd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 -- 172.21.15.83:0/1854938466 >> 172.21.15.83:0/1854938466 conn(0x7f31e80cf300 msgr2=0x7f31e80cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 -- 172.21.15.83:0/1854938466 shutdown_connections 2023-11-28T11:09:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.896+0000 7f31ef5e3700 1 -- 172.21.15.83:0/1854938466 wait complete. 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.897+0000 7f31ef5e3700 1 Processor -- start 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.897+0000 7f31ef5e3700 1 -- start start 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31e80ca060 0x7f31e80c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 0x7f31e80c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31e815da70 0x7f31e80c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81f3e70 con 0x7f31e80cced0 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e806b870 con 0x7f31e80ca060 2023-11-28T11:09:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c71a0 con 0x7f31e815da70 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 0x7f31e80c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 0x7f31e80c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50030/0 (socket says 172.21.15.83:50030) 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 -- 172.21.15.83:0/2119726824 learned_addr learned my addr 172.21.15.83:0/2119726824 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ed37f700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31e80ca060 0x7f31e80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31edb80700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31e815da70 0x7f31e80c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 -- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31e815da70 msgr2=0x7f31e80c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31e815da70 0x7f31e80c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 -- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31e80ca060 msgr2=0x7f31e80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31e80ca060 0x7f31e80c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 -- 172.21.15.83:0/2119726824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8009550 con 0x7f31e80cced0 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ecb7e700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 0x7f31e80c65e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c4e0 tx=0x7f31e400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40052d0 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4005430 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e8071e70 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.898+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40038c0 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.899+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e80ce0c0 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.899+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e80cebe0 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.900+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f31e4019030 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.900+0000 7f31ddffb700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31cc0682f0 0x7f31cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.900+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f31e4085250 con 0x7f31e80cced0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.900+0000 7f31ed37f700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31cc0682f0 0x7f31cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.900+0000 7f31ed37f700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31cc0682f0 0x7f31cc06a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f31d8014040 tx=0x7f31d8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:25.904+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f31e4051780 con 0x7f31e80cced0 2023-11-28T11:09:26.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.090+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31e81f3fb0 con 0x7f31e80cced0 2023-11-28T11:09:26.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.090+0000 7f31ddffb700 1 -- 172.21.15.83:0/2119726824 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31e81f3fb0 con 0x7f31e80cced0 2023-11-28T11:09:26.092 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:26.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31cc0682f0 msgr2=0x7f31cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31cc0682f0 0x7f31cc06a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f31d8014040 tx=0x7f31d8008030 comp rx=0 tx=0).stop 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 msgr2=0x7f31e80c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 0x7f31e80c65e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c4e0 tx=0x7f31e400c7f0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 shutdown_connections 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31e815da70 0x7f31e80c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31e80ca060 0x7f31e80c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31cc0682f0 0x7f31cc06a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 --2- 172.21.15.83:0/2119726824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e80cced0 0x7f31e80c65e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 >> 172.21.15.83:0/2119726824 conn(0x7f31e80cf300 msgr2=0x7f31e80aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 shutdown_connections 2023-11-28T11:09:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.092+0000 7f31ef5e3700 1 -- 172.21.15.83:0/2119726824 wait complete. 2023-11-28T11:09:26.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:26.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.226+0000 7f65f9e78700 1 Processor -- start 2023-11-28T11:09:26.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.226+0000 7f65f9e78700 1 -- start start 2023-11-28T11:09:26.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.226+0000 7f65f9e78700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f65f415cd30 0x7f65f40cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.226+0000 7f65f9e78700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f40b23b0 con 0x7f65f40b1f50 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.226+0000 7f65f9e78700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f406cad0 con 0x7f65f40cb200 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.226+0000 7f65f9e78700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f40cb670 con 0x7f65f415cd30 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f3fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f65f40b1f50 0x7f65f40cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47900/0 (socket says 172.21.15.83:47900) 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f3fff700 1 -- 172.21.15.83:0/2621890547 learned_addr learned my addr 172.21.15.83:0/2621890547 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f37fe700 1 --2- 172.21.15.83:0/2621890547 >> v2:172.21.15.90:3300/0 conn(0x7f65f415cd30 0x7f65f40cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621506057 0 0) 0x7f65f40b23b0 con 0x7f65f40b1f50 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65dc0035b0 con 0x7f65f40b1f50 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196175437 0 0) 0x7f65dc0035b0 con 0x7f65f40b1f50 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 >> v2:172.21.15.90:3300/0 conn(0x7f65f415cd30 msgr2=0x7f65f40cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 --2- 172.21.15.83:0/2621890547 >> v2:172.21.15.90:3300/0 conn(0x7f65f415cd30 0x7f65f40cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 >> v1:172.21.15.90:6789/0 conn(0x7f65f40cb200 legacy=0x7f65f40b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f40af4a0 con 0x7f65f40b1f50 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f65ec0019f0 con 0x7f65f40b1f50 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 >> v1:172.21.15.83:6789/0 conn(0x7f65f40b1f50 legacy=0x7f65f40cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65dc003a00 0x7f65dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65dc005200 0x7f65dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40cb670 con 0x7f65dc004850 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00c080 con 0x7f65dc003a00 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00c240 con 0x7f65dc005200 2023-11-28T11:09:26.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f37fe700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65dc003a00 0x7f65dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.227+0000 7f65f3fff700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65dc005200 0x7f65dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f2ffd700 1 -- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65dc005200 msgr2=0x7f65dc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65dc005200 0x7f65dc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f2ffd700 1 -- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65dc003a00 msgr2=0x7f65dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65dc003a00 0x7f65dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f3fff700 1 -- 172.21.15.83:0/2621890547 reap_dead start 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f2ffd700 1 -- 172.21.15.83:0/2621890547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0035b0 con 0x7f65dc004850 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65dc004cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65e80017c0 tx=0x7f65dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5aa9a4550b750 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e80052d0 con 0x7f65dc004850 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8005450 con 0x7f65dc004850 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f27fc700 1 -- 172.21.15.83:0/2621890547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e80038c0 con 0x7f65dc004850 2023-11-28T11:09:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 -- 172.21.15.83:0/2621890547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 msgr2=0x7f65dc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65dc004cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65e80017c0 tx=0x7f65dc005880 comp rx=0 tx=0).stop 2023-11-28T11:09:26.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 -- 172.21.15.83:0/2621890547 shutdown_connections 2023-11-28T11:09:26.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 --2- 172.21.15.83:0/2621890547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65dc004cc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 -- 172.21.15.83:0/2621890547 >> 172.21.15.83:0/2621890547 conn(0x7f65f40cc690 msgr2=0x7f65f40cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 -- 172.21.15.83:0/2621890547 shutdown_connections 2023-11-28T11:09:26.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.228+0000 7f65f9e78700 1 -- 172.21.15.83:0/2621890547 wait complete. 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.230+0000 7f65f9e78700 1 Processor -- start 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.230+0000 7f65f9e78700 1 -- start start 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f9e78700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65f41f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f9e78700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65f415a920 0x7f65f415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f9e78700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f40ad510 0x7f65f40ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f9e78700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f415b2b0 con 0x7f65dc004850 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f9e78700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40ae790 con 0x7f65f40ad510 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f9e78700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40ae1d0 con 0x7f65f415a920 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65f41f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65f41f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50048/0 (socket says 172.21.15.83:50048) 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 -- 172.21.15.83:0/508544863 learned_addr learned my addr 172.21.15.83:0/508544863 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f3fff700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f40ad510 0x7f65f40ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65f415a920 0x7f65f415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 -- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65f415a920 msgr2=0x7f65f415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65f415a920 0x7f65f415ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 -- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f40ad510 msgr2=0x7f65f40ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f40ad510 0x7f65f40ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.231+0000 7f65f37fe700 1 -- 172.21.15.83:0/508544863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f406c1a0 con 0x7f65dc004850 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.232+0000 7f65f37fe700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65f41f19e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f65e40119d0 tx=0x7f65e4011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.232+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4009630 con 0x7f65dc004850 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.232+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4004240 con 0x7f65dc004850 2023-11-28T11:09:26.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.232+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f406bec0 con 0x7f65dc004850 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.232+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400f730 con 0x7f65dc004850 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.232+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f406c480 con 0x7f65dc004850 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.233+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f40742b0 con 0x7f65dc004850 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.233+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65e4025070 con 0x7f65dc004850 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.233+0000 7f65dbfff700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e0068320 0x7f65e006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.233+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f65e4086470 con 0x7f65dc004850 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.233+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e0068320 0x7f65e006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.234+0000 7f65f2ffd700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e0068320 0x7f65e006a7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000c00 tx=0x7f65e8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.238+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65e40529a0 con 0x7f65dc004850 2023-11-28T11:09:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.422+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65f40b2570 con 0x7f65dc004850 2023-11-28T11:09:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.423+0000 7f65dbfff700 1 -- 172.21.15.83:0/508544863 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65e4055fc0 con 0x7f65dc004850 2023-11-28T11:09:26.426 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e0068320 msgr2=0x7f65e006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e0068320 0x7f65e006a7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000c00 tx=0x7f65e8009030 comp rx=0 tx=0).stop 2023-11-28T11:09:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 msgr2=0x7f65f41f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65f41f19e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f65e40119d0 tx=0x7f65e4011d90 comp rx=0 tx=0).stop 2023-11-28T11:09:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 shutdown_connections 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f40ad510 0x7f65f40ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e0068320 0x7f65e006a7d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65dc004850 0x7f65f41f19e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 --2- 172.21.15.83:0/508544863 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65f415a920 0x7f65f415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 >> 172.21.15.83:0/508544863 conn(0x7f65f40cc690 msgr2=0x7f65f40cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 shutdown_connections 2023-11-28T11:09:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.425+0000 7f65f9e78700 1 -- 172.21.15.83:0/508544863 wait complete. 2023-11-28T11:09:26.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.560+0000 7f7fd8ce3700 1 Processor -- start 2023-11-28T11:09:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.560+0000 7f7fd8ce3700 1 -- start start 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd8ce3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fd415c890 0x7f7fd40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd8ce3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7fd40b4bb0 0x7f7fd406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd8ce3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd406d4a0 con 0x7f7fd406e1a0 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd8ce3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd406e620 con 0x7f7fd415c890 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd8ce3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd40b5020 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fd415c890 0x7f7fd40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fd415c890 0x7f7fd40aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50056/0 (socket says 172.21.15.83:50056) 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 -- 172.21.15.83:0/2950956138 learned_addr learned my addr 172.21.15.83:0/2950956138 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd1d9b700 1 --2- 172.21.15.83:0/2950956138 >> v2:172.21.15.90:3300/0 conn(0x7f7fd40b4bb0 0x7f7fd406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 -- 172.21.15.83:0/2950956138 >> v2:172.21.15.90:3300/0 conn(0x7f7fd40b4bb0 msgr2=0x7f7fd406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 --2- 172.21.15.83:0/2950956138 >> v2:172.21.15.90:3300/0 conn(0x7f7fd40b4bb0 0x7f7fd406dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 -- 172.21.15.83:0/2950956138 >> v1:172.21.15.83:6789/0 conn(0x7f7fd406e1a0 legacy=0x7f7fd41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 -- 172.21.15.83:0/2950956138 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd406c160 con 0x7f7fd415c890 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd1d9b700 1 --2- 172.21.15.83:0/2950956138 >> v2:172.21.15.90:3300/0 conn(0x7f7fd40b4bb0 0x7f7fd406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.561+0000 7f7fd259c700 1 --2- 172.21.15.83:0/2950956138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415c890 0x7f7fd40aee60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4071890 tx=0x7f7fc4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b42d6a27561744 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd159a700 1 -- 172.21.15.83:0/2950956138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4005e30 con 0x7f7fd415c890 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd159a700 1 -- 172.21.15.83:0/2950956138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc4004030 con 0x7f7fd415c890 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd159a700 1 -- 172.21.15.83:0/2950956138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40043e0 con 0x7f7fd415c890 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2950956138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415c890 msgr2=0x7f7fd40aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2950956138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415c890 0x7f7fd40aee60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4071890 tx=0x7f7fc4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2950956138 shutdown_connections 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2950956138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415c890 0x7f7fd40aee60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2950956138 >> v2:172.21.15.90:3300/0 conn(0x7f7fd40b4bb0 0x7f7fd406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2950956138 >> 172.21.15.83:0/2950956138 conn(0x7f7fd40b2b60 msgr2=0x7f7fd40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2950956138 shutdown_connections 2023-11-28T11:09:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.562+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2950956138 wait complete. 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 Processor -- start 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 -- start start 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fd406e1a0 0x7f7fd415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 0x7f7fd414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fd415c890 0x7f7fd40c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415a960 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415ac40 con 0x7f7fd406e1a0 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.564+0000 7f7fd8ce3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415af20 con 0x7f7fd415c890 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 0x7f7fd414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 0x7f7fd414f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50064/0 (socket says 172.21.15.83:50064) 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 -- 172.21.15.83:0/2464588991 learned_addr learned my addr 172.21.15.83:0/2464588991 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd259c700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fd406e1a0 0x7f7fd415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd2d9d700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fd415c890 0x7f7fd40c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 -- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fd415c890 msgr2=0x7f7fd40c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fd415c890 0x7f7fd40c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 -- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fd406e1a0 msgr2=0x7f7fd415ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fd406e1a0 0x7f7fd415ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 -- 172.21.15.83:0/2464588991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009500 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd1d9b700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 0x7f7fd414f1a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800e810 tx=0x7f7fc800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800c020 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc8004250 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd406c400 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80036a0 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.565+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd40b5ae0 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.566+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd40b5240 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.567+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7fc8022070 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.567+0000 7f7fc2ffd700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7fb40682f0 0x7f7fb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.567+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7fc8085d50 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.567+0000 7f7fd259c700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7fb40682f0 0x7f7fb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.567+0000 7f7fd259c700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7fb40682f0 0x7f7fb406a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4071890 tx=0x7f7fc4008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.571+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7fc8052280 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.753+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7fd40742b0 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.753+0000 7f7fc2ffd700 1 -- 172.21.15.83:0/2464588991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7fc800f040 con 0x7f7fd40b4bb0 2023-11-28T11:09:26.755 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7fb40682f0 msgr2=0x7f7fb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7fb40682f0 0x7f7fb406a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4071890 tx=0x7f7fc4008070 comp rx=0 tx=0).stop 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 msgr2=0x7f7fd414f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 0x7f7fd414f1a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800e810 tx=0x7f7fc800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 shutdown_connections 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fd415c890 0x7f7fd40c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fd406e1a0 0x7f7fd415ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7fb40682f0 0x7f7fb406a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 --2- 172.21.15.83:0/2464588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40b4bb0 0x7f7fd414f1a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 >> 172.21.15.83:0/2464588991 conn(0x7f7fd40b2b60 msgr2=0x7f7fd40b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 shutdown_connections 2023-11-28T11:09:26.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.755+0000 7f7fd8ce3700 1 -- 172.21.15.83:0/2464588991 wait complete. 2023-11-28T11:09:26.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:26.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.888+0000 7f49ba78a700 1 Processor -- start 2023-11-28T11:09:26.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.888+0000 7f49ba78a700 1 -- start start 2023-11-28T11:09:26.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49ba78a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 0x7f49b415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49ba78a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f49b40d0ae0 0x7f49b40d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49ba78a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b4151a30 con 0x7f49b415e5c0 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49ba78a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b40d14a0 con 0x7f49b40d0ae0 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49ba78a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40cb460 con 0x7f49b40ce1b0 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49b37fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 0x7f49b415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49b8d27700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f49b40d0ae0 0x7f49b40d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49b37fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 0x7f49b415b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:34274/0 (socket says 172.21.15.83:34274) 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.889+0000 7f49b37fe700 1 -- 172.21.15.83:0/2092471521 learned_addr learned my addr 172.21.15.83:0/2092471521 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b37fe700 1 -- 172.21.15.83:0/2092471521 >> v2:172.21.15.90:3300/0 conn(0x7f49b40d0ae0 msgr2=0x7f49b40d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b37fe700 1 --2- 172.21.15.83:0/2092471521 >> v2:172.21.15.90:3300/0 conn(0x7f49b40d0ae0 0x7f49b40d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b37fe700 1 -- 172.21.15.83:0/2092471521 >> v1:172.21.15.90:6789/0 conn(0x7f49b415e5c0 legacy=0x7f49b415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b37fe700 1 -- 172.21.15.83:0/2092471521 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b40cb740 con 0x7f49b40ce1b0 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b8d27700 1 --2- 172.21.15.83:0/2092471521 >> v2:172.21.15.90:3300/0 conn(0x7f49b40d0ae0 0x7f49b40d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b37fe700 1 --2- 172.21.15.83:0/2092471521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 0x7f49b415b3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49a40098f0 tx=0x7f49a4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7791ae0a1ee465 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b2ffd700 1 -- 172.21.15.83:0/2092471521 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4011830 con 0x7f49b40ce1b0 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b2ffd700 1 -- 172.21.15.83:0/2092471521 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a4011990 con 0x7f49b40ce1b0 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.890+0000 7f49b2ffd700 1 -- 172.21.15.83:0/2092471521 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4011d20 con 0x7f49b40ce1b0 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 -- 172.21.15.83:0/2092471521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 msgr2=0x7f49b415b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2092471521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 0x7f49b415b3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49a40098f0 tx=0x7f49a4009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 -- 172.21.15.83:0/2092471521 shutdown_connections 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2092471521 >> v2:172.21.15.90:3300/0 conn(0x7f49b40d0ae0 0x7f49b40d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2092471521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b40ce1b0 0x7f49b415b3b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 -- 172.21.15.83:0/2092471521 >> 172.21.15.83:0/2092471521 conn(0x7f49b406ea50 msgr2=0x7f49b4070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 -- 172.21.15.83:0/2092471521 shutdown_connections 2023-11-28T11:09:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.891+0000 7f49ba78a700 1 -- 172.21.15.83:0/2092471521 wait complete. 2023-11-28T11:09:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.892+0000 7f49ba78a700 1 Processor -- start 2023-11-28T11:09:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.892+0000 7f49ba78a700 1 -- start start 2023-11-28T11:09:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49ba78a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b40d0ae0 0x7f49b4145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49ba78a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b415e5c0 0x7f49b415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49ba78a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 0x7f49b415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49ba78a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b414bdf0 con 0x7f49b41e4eb0 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49ba78a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b414b170 con 0x7f49b40d0ae0 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49ba78a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b414b450 con 0x7f49b415e5c0 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 0x7f49b415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 0x7f49b415da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50080/0 (socket says 172.21.15.83:50080) 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b37fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b415e5c0 0x7f49b415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 -- 172.21.15.83:0/2690975287 learned_addr learned my addr 172.21.15.83:0/2690975287 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b40d0ae0 0x7f49b4145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 -- 172.21.15.83:0/2690975287 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b415e5c0 msgr2=0x7f49b415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b415e5c0 0x7f49b415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 -- 172.21.15.83:0/2690975287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b40d0ae0 msgr2=0x7f49b4145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b40d0ae0 0x7f49b4145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 -- 172.21.15.83:0/2690975287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac009a70 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.893+0000 7f49b8d27700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 0x7f49b415da00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00efc0 tx=0x7f49ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.894+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00ca90 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.894+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac00f040 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.894+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a4009550 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.894+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00ce30 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.894+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b415ae90 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.894+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b415abb0 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.895+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f49ac014480 con 0x7f49b41e4eb0 2023-11-28T11:09:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.895+0000 7f49b0ff9700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49940682f0 0x7f499406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.896+0000 7f49b3fff700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49940682f0 0x7f499406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.896+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f49ac086830 con 0x7f49b41e4eb0 2023-11-28T11:09:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.896+0000 7f49b3fff700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49940682f0 0x7f499406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f499c007fd0 tx=0x7f499c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:26.900+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49ac052c80 con 0x7f49b41e4eb0 2023-11-28T11:09:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.097+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f49b415b8f0 con 0x7f49b41e4eb0 2023-11-28T11:09:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.098+0000 7f49b0ff9700 1 -- 172.21.15.83:0/2690975287 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f49ac009cd0 con 0x7f49b41e4eb0 2023-11-28T11:09:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49940682f0 msgr2=0x7f499406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49940682f0 0x7f499406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f499c007fd0 tx=0x7f499c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 msgr2=0x7f49b415da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 0x7f49b415da00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00efc0 tx=0x7f49ac00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 shutdown_connections 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49940682f0 0x7f499406a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b41e4eb0 0x7f49b415da00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b40d0ae0 0x7f49b4145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 --2- 172.21.15.83:0/2690975287 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49b415e5c0 0x7f49b415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 >> 172.21.15.83:0/2690975287 conn(0x7f49b40ce1b0 msgr2=0x7f49b4150200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 shutdown_connections 2023-11-28T11:09:27.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.100+0000 7f49ba78a700 1 -- 172.21.15.83:0/2690975287 wait complete. 2023-11-28T11:09:27.103 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:09:27.113 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T11:09:27.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:27.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T11:09:27.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:09:27.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:27.152 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:09:27.209 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:27.209 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:27.230 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:27.230 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:27.292 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:09:27.292 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:27.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.470+0000 7fb8c388b700 1 Processor -- start 2023-11-28T11:09:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.470+0000 7fb8c388b700 1 -- start start 2023-11-28T11:09:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c388b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c388b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb8bc15a820 0x7fb8bc0ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c388b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8bc159ad0 0x7fb8bc0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c388b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc06b670 con 0x7fb8bc159ad0 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c388b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc06aa60 con 0x7fb8bc15a820 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c388b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc06ad40 con 0x7fb8bc15a450 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c1e28700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8bc159ad0 0x7fb8bc0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c1e28700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8bc159ad0 0x7fb8bc0cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50088/0 (socket says 172.21.15.83:50088) 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c1e28700 1 -- 172.21.15.83:0/1291407979 learned_addr learned my addr 172.21.15.83:0/1291407979 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1627700 1 --2- 172.21.15.83:0/1291407979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.471+0000 7fb8c0e26700 1 --2- 172.21.15.83:0/1291407979 >> v2:172.21.15.90:3300/0 conn(0x7fb8bc15a820 0x7fb8bc0ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1e28700 1 -- 172.21.15.83:0/1291407979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 msgr2=0x7fb8bc06a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1e28700 1 --2- 172.21.15.83:0/1291407979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc06a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1e28700 1 -- 172.21.15.83:0/1291407979 >> v2:172.21.15.90:3300/0 conn(0x7fb8bc15a820 msgr2=0x7fb8bc0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1e28700 1 --2- 172.21.15.83:0/1291407979 >> v2:172.21.15.90:3300/0 conn(0x7fb8bc15a820 0x7fb8bc0ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1e28700 1 -- 172.21.15.83:0/1291407979 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc06b020 con 0x7fb8bc159ad0 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c1e28700 1 --2- 172.21.15.83:0/1291407979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc0cad90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0724b0 tx=0x7fb8b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7c3643cbaee7b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8b3fff700 1 -- 172.21.15.83:0/1291407979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400e070 con 0x7fb8bc159ad0 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8b3fff700 1 -- 172.21.15.83:0/1291407979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b400b9c0 con 0x7fb8bc159ad0 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8b3fff700 1 -- 172.21.15.83:0/1291407979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400bd70 con 0x7fb8bc159ad0 2023-11-28T11:09:27.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 -- 172.21.15.83:0/1291407979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc159ad0 msgr2=0x7fb8bc0cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 --2- 172.21.15.83:0/1291407979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc0cad90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0724b0 tx=0x7fb8b400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 -- 172.21.15.83:0/1291407979 shutdown_connections 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 --2- 172.21.15.83:0/1291407979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc06a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 --2- 172.21.15.83:0/1291407979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc0cad90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 --2- 172.21.15.83:0/1291407979 >> v2:172.21.15.90:3300/0 conn(0x7fb8bc15a820 0x7fb8bc0ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.472+0000 7fb8c388b700 1 -- 172.21.15.83:0/1291407979 >> 172.21.15.83:0/1291407979 conn(0x7fb8bc0cf220 msgr2=0x7fb8bc0cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.473+0000 7fb8c388b700 1 -- 172.21.15.83:0/1291407979 shutdown_connections 2023-11-28T11:09:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.473+0000 7fb8c388b700 1 -- 172.21.15.83:0/1291407979 wait complete. 2023-11-28T11:09:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 Processor -- start 2023-11-28T11:09:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 -- start start 2023-11-28T11:09:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc09e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 0x7fb8bc0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc09f0c0 con 0x7fb8bc09ec40 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc1565f0 con 0x7fb8bc159ad0 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.474+0000 7fb8c388b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc09f2c0 con 0x7fb8bc15a450 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 0x7fb8bc0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 0x7fb8bc0c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50094/0 (socket says 172.21.15.83:50094) 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 -- 172.21.15.83:0/3272549853 learned_addr learned my addr 172.21.15.83:0/3272549853 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1627700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c0e26700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 -- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 msgr2=0x7fb8bc09e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc09e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 -- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc159ad0 msgr2=0x7fb8bc1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 -- 172.21.15.83:0/3272549853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4009b30 con 0x7fb8bc09ec40 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c1e28700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 0x7fb8bc0c8500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0ced00 tx=0x7fb8b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40092e0 con 0x7fb8bc09ec40 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b4004250 con 0x7fb8bc09ec40 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc0cb550 con 0x7fb8bc09ec40 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4004620 con 0x7fb8bc09ec40 2023-11-28T11:09:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.475+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc06b300 con 0x7fb8bc09ec40 2023-11-28T11:09:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.476+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc156760 con 0x7fb8bc09ec40 2023-11-28T11:09:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.477+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb8b402c070 con 0x7fb8bc09ec40 2023-11-28T11:09:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.477+0000 7fb8b1ffb700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb8a00682f0 0x7fb8a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.477+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb8b4087640 con 0x7fb8bc09ec40 2023-11-28T11:09:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.477+0000 7fb8c1627700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb8a00682f0 0x7fb8a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.478+0000 7fb8c1627700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb8a00682f0 0x7fb8a006a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0ce0a0 tx=0x7fb8ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.482+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb8b4053b70 con 0x7fb8bc09ec40 2023-11-28T11:09:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.670+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8bc074240 con 0x7fb8bc09ec40 2023-11-28T11:09:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.671+0000 7fb8b1ffb700 1 -- 172.21.15.83:0/3272549853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19737 (secure 0 0 0) 0x7fb8b4004780 con 0x7fb8bc09ec40 2023-11-28T11:09:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:27.673 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:25.004035+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb8a00682f0 msgr2=0x7fb8a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb8a00682f0 0x7fb8a006a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0ce0a0 tx=0x7fb8ac008040 comp rx=0 tx=0).stop 2023-11-28T11:09:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 msgr2=0x7fb8bc0c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 0x7fb8bc0c8500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0ced00 tx=0x7fb8b4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 shutdown_connections 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb8a00682f0 0x7fb8a006a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8bc09ec40 0x7fb8bc0c8500 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc159ad0 0x7fb8bc1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 --2- 172.21.15.83:0/3272549853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8bc15a450 0x7fb8bc09e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 >> 172.21.15.83:0/3272549853 conn(0x7fb8bc0cf220 msgr2=0x7fb8bc06d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 shutdown_connections 2023-11-28T11:09:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.673+0000 7fb8c388b700 1 -- 172.21.15.83:0/3272549853 wait complete. 2023-11-28T11:09:27.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:27.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.810+0000 7fc744950700 1 Processor -- start 2023-11-28T11:09:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.810+0000 7fc744950700 1 -- start start 2023-11-28T11:09:27.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc744950700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc740158bc0 0x7fc7400cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc744950700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7400cb680 0x7fc740071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc744950700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc740158f90 con 0x7fc740071880 2023-11-28T11:09:27.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc744950700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc740071d20 con 0x7fc740158bc0 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc744950700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74015c840 con 0x7fc7400cb680 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc740158bc0 0x7fc7400cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc740158bc0 0x7fc7400cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50098/0 (socket says 172.21.15.83:50098) 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 -- 172.21.15.83:0/480127495 learned_addr learned my addr 172.21.15.83:0/480127495 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73dd9b700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7400cb680 0x7fc740071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 -- 172.21.15.83:0/480127495 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7400cb680 msgr2=0x7fc740071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7400cb680 0x7fc740071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 -- 172.21.15.83:0/480127495 >> v1:172.21.15.90:6789/0 conn(0x7fc740071880 legacy=0x7fc74006e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 -- 172.21.15.83:0/480127495 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74015cb20 con 0x7fc740158bc0 2023-11-28T11:09:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73dd9b700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7400cb680 0x7fc740071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73e59c700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400cdd70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7300098f0 tx=0x7fc730009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ed828d4879f7e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73d59a700 1 -- 172.21.15.83:0/480127495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000b8a0 con 0x7fc740158bc0 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.811+0000 7fc73d59a700 1 -- 172.21.15.83:0/480127495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000ba00 con 0x7fc740158bc0 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc73d59a700 1 -- 172.21.15.83:0/480127495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000bd90 con 0x7fc740158bc0 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 -- 172.21.15.83:0/480127495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 msgr2=0x7fc7400cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400cdd70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7300098f0 tx=0x7fc730009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 -- 172.21.15.83:0/480127495 shutdown_connections 2023-11-28T11:09:27.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400cdd70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 --2- 172.21.15.83:0/480127495 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7400cb680 0x7fc740071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 -- 172.21.15.83:0/480127495 >> 172.21.15.83:0/480127495 conn(0x7fc7400cf300 msgr2=0x7fc7400cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 -- 172.21.15.83:0/480127495 shutdown_connections 2023-11-28T11:09:27.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.812+0000 7fc744950700 1 -- 172.21.15.83:0/480127495 wait complete. 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.813+0000 7fc744950700 1 Processor -- start 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 -- start start 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc740071880 0x7fc74014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7400cb680 0x7fc7401511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740159440 con 0x7fc740158bc0 2023-11-28T11:09:27.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74015b250 con 0x7fc7400cb680 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc744950700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74015aa40 con 0x7fc740071880 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc73ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc73ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50104/0 (socket says 172.21.15.83:50104) 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc73ed9d700 1 -- 172.21.15.83:0/92252820 learned_addr learned my addr 172.21.15.83:0/92252820 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.814+0000 7fc73dd9b700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7400cb680 0x7fc7401511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73e59c700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc740071880 0x7fc74014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73ed9d700 1 -- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc740071880 msgr2=0x7fc74014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73ed9d700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc740071880 0x7fc74014ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73ed9d700 1 -- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7400cb680 msgr2=0x7fc7401511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73ed9d700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7400cb680 0x7fc7401511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73ed9d700 1 -- 172.21.15.83:0/92252820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730009550 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc73ed9d700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400acee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc73800cc50 tx=0x7fc73800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738003c00 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc738003d60 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74015d0e0 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800aab0 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.815+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74015ce00 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.816+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74015d3c0 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.816+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc738010050 con 0x7fc740158bc0 2023-11-28T11:09:27.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.817+0000 7fc72effd700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc7200682f0 0x7fc72006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.817+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc738085ee0 con 0x7fc740158bc0 2023-11-28T11:09:27.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.817+0000 7fc73e59c700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc7200682f0 0x7fc72006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.817+0000 7fc73e59c700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc7200682f0 0x7fc72006a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc730014040 tx=0x7fc730008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:27.821+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc738052320 con 0x7fc740158bc0 2023-11-28T11:09:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.002+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7400742b0 con 0x7fc740158bc0 2023-11-28T11:09:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.002+0000 7fc72effd700 1 -- 172.21.15.83:0/92252820 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc738055940 con 0x7fc740158bc0 2023-11-28T11:09:28.004 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc7200682f0 msgr2=0x7fc72006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc7200682f0 0x7fc72006a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc730014040 tx=0x7fc730008030 comp rx=0 tx=0).stop 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 msgr2=0x7fc7400acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400acee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc73800cc50 tx=0x7fc73800cf60 comp rx=0 tx=0).stop 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 shutdown_connections 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc7200682f0 0x7fc72006a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc740158bc0 0x7fc7400acee0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc740071880 0x7fc74014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 --2- 172.21.15.83:0/92252820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7400cb680 0x7fc7401511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 >> 172.21.15.83:0/92252820 conn(0x7fc7400cf300 msgr2=0x7fc7400aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 shutdown_connections 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.004+0000 7fc744950700 1 -- 172.21.15.83:0/92252820 wait complete. 2023-11-28T11:09:28.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.140+0000 7f9a3a755700 1 Processor -- start 2023-11-28T11:09:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.140+0000 7f9a3a755700 1 -- start start 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a3a755700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a3a755700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a340b4bb0 0x7f9a3406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a3a755700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9a3406e1a0 0x7f9a341604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a3a755700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3406e620 con 0x7f9a340b4bb0 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a3a755700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340b5020 con 0x7f9a3406e1a0 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a3a755700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3406d4a0 con 0x7f9a3415c890 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a340b4bb0 0x7f9a3406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a340b4bb0 0x7f9a3406dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50112/0 (socket says 172.21.15.83:50112) 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 -- 172.21.15.83:0/2486265359 learned_addr learned my addr 172.21.15.83:0/2486265359 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 -- 172.21.15.83:0/2486265359 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 msgr2=0x7f9a340aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 --2- 172.21.15.83:0/2486265359 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 -- 172.21.15.83:0/2486265359 >> v2:172.21.15.90:3300/0 conn(0x7f9a3406e1a0 msgr2=0x7f9a341604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 --2- 172.21.15.83:0/2486265359 >> v2:172.21.15.90:3300/0 conn(0x7f9a3406e1a0 0x7f9a341604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 -- 172.21.15.83:0/2486265359 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340b5ae0 con 0x7f9a340b4bb0 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a337fe700 1 --2- 172.21.15.83:0/2486265359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3406dc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a240098f0 tx=0x7f9a24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed36cd661cf656f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a32ffd700 1 -- 172.21.15.83:0/2486265359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2400b860 con 0x7f9a340b4bb0 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a32ffd700 1 -- 172.21.15.83:0/2486265359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2400b9c0 con 0x7f9a340b4bb0 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.141+0000 7f9a32ffd700 1 -- 172.21.15.83:0/2486265359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2400bd50 con 0x7f9a340b4bb0 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 -- 172.21.15.83:0/2486265359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 msgr2=0x7f9a3406dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2486265359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3406dc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a240098f0 tx=0x7f9a24009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 -- 172.21.15.83:0/2486265359 shutdown_connections 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2486265359 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2486265359 >> v2:172.21.15.90:3300/0 conn(0x7f9a3406e1a0 0x7f9a341604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2486265359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3406dc60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 -- 172.21.15.83:0/2486265359 >> 172.21.15.83:0/2486265359 conn(0x7f9a340b2b60 msgr2=0x7f9a340b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 -- 172.21.15.83:0/2486265359 shutdown_connections 2023-11-28T11:09:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.142+0000 7f9a3a755700 1 -- 172.21.15.83:0/2486265359 wait complete. 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.143+0000 7f9a3a755700 1 Processor -- start 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.143+0000 7f9a3a755700 1 -- start start 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a3a755700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a3406e1a0 0x7f9a3409d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a3a755700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a3a755700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a3a755700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3415f030 con 0x7f9a340b4bb0 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a3a755700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3415e160 con 0x7f9a3406e1a0 2023-11-28T11:09:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a3a755700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340cd180 con 0x7f9a3415c890 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50118/0 (socket says 172.21.15.83:50118) 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a337fe700 1 -- 172.21.15.83:0/2824939852 learned_addr learned my addr 172.21.15.83:0/2824939852 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a38cf2700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a33fff700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a3406e1a0 0x7f9a3409d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a337fe700 1 -- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 msgr2=0x7f9a340c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.144+0000 7f9a337fe700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a337fe700 1 -- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a3406e1a0 msgr2=0x7f9a3409d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a337fe700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a3406e1a0 0x7f9a3409d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a337fe700 1 -- 172.21.15.83:0/2824939852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a24009550 con 0x7f9a340b4bb0 2023-11-28T11:09:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a33fff700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a3406e1a0 0x7f9a3409d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a337fe700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3415f450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a24014040 tx=0x7f9a24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240036a0 con 0x7f9a340b4bb0 2023-11-28T11:09:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2401e070 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a340b5240 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a24003a40 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.145+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3406c400 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.146+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a340b5520 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.146+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a2400f050 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.146+0000 7f9a30ff9700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a200682f0 0x7f9a2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.147+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9a24020070 con 0x7f9a340b4bb0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.147+0000 7f9a33fff700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a200682f0 0x7f9a2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.147+0000 7f9a33fff700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a200682f0 0x7f9a2006a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006fd0 tx=0x7f9a28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.151+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a24053aa0 con 0x7f9a340b4bb0 2023-11-28T11:09:28.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.330+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a340742b0 con 0x7f9a340b4bb0 2023-11-28T11:09:28.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.331+0000 7f9a30ff9700 1 -- 172.21.15.83:0/2824939852 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a24023380 con 0x7f9a340b4bb0 2023-11-28T11:09:28.333 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a200682f0 msgr2=0x7f9a2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a200682f0 0x7f9a2006a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006fd0 tx=0x7f9a28008040 comp rx=0 tx=0).stop 2023-11-28T11:09:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 msgr2=0x7f9a3415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3415f450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a24014040 tx=0x7f9a24004750 comp rx=0 tx=0).stop 2023-11-28T11:09:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 shutdown_connections 2023-11-28T11:09:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a3415c890 0x7f9a340c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a3406e1a0 0x7f9a3409d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9a200682f0 0x7f9a2006a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 --2- 172.21.15.83:0/2824939852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a340b4bb0 0x7f9a3415f450 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 >> 172.21.15.83:0/2824939852 conn(0x7f9a340b2b60 msgr2=0x7f9a3415c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 shutdown_connections 2023-11-28T11:09:28.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.333+0000 7f9a3a755700 1 -- 172.21.15.83:0/2824939852 wait complete. 2023-11-28T11:09:28.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:28.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.467+0000 7f5a1ca9e700 1 Processor -- start 2023-11-28T11:09:28.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.467+0000 7f5a1ca9e700 1 -- start start 2023-11-28T11:09:28.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.468+0000 7f5a1ca9e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5a180b1690 0x7f5a180cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.468+0000 7f5a1ca9e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a180cb3a0 con 0x7f5a180b5680 2023-11-28T11:09:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.468+0000 7f5a1ca9e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a180cb640 con 0x7f5a18159a20 2023-11-28T11:09:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.468+0000 7f5a1ca9e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a180b54f0 con 0x7f5a180b1690 2023-11-28T11:09:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.468+0000 7f5a16d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5a180b5680 0x7f5a1806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47986/0 (socket says 172.21.15.83:47986) 2023-11-28T11:09:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a16d9d700 1 -- 172.21.15.83:0/255300784 learned_addr learned my addr 172.21.15.83:0/255300784 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1659c700 1 --2- 172.21.15.83:0/255300784 >> v2:172.21.15.90:3300/0 conn(0x7f5a180b1690 0x7f5a180cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142185336 0 0) 0x7f5a180cb3a0 con 0x7f5a180b5680 2023-11-28T11:09:28.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a000035b0 con 0x7f5a180b5680 2023-11-28T11:09:28.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16169411 0 0) 0x7f5a000035b0 con 0x7f5a180b5680 2023-11-28T11:09:28.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 >> v2:172.21.15.90:3300/0 conn(0x7f5a180b1690 msgr2=0x7f5a180cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 --2- 172.21.15.83:0/255300784 >> v2:172.21.15.90:3300/0 conn(0x7f5a180b1690 0x7f5a180cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 >> v1:172.21.15.90:6789/0 conn(0x7f5a18159a20 legacy=0x7f5a180b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180cb920 con 0x7f5a180b5680 2023-11-28T11:09:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f5a100019f0 con 0x7f5a180b5680 2023-11-28T11:09:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 >> v1:172.21.15.83:6789/0 conn(0x7f5a180b5680 legacy=0x7f5a1806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a00003a00 0x7f5a00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00004850 0x7f5a00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a00005200 0x7f5a0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0000c0e0 con 0x7f5a00005200 2023-11-28T11:09:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0000c250 con 0x7f5a00004850 2023-11-28T11:09:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0000c410 con 0x7f5a00003a00 2023-11-28T11:09:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a16d9d700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a00005200 0x7f5a0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a15d9b700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00004850 0x7f5a00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.469+0000 7f5a1659c700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a00003a00 0x7f5a00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 -- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a00003a00 msgr2=0x7f5a00003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a00003a00 0x7f5a00003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 -- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00004850 msgr2=0x7f5a00004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00004850 0x7f5a00004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 -- 172.21.15.83:0/255300784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a000035b0 con 0x7f5a00005200 2023-11-28T11:09:28.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 -- 172.21.15.83:0/255300784 reap_dead start 2023-11-28T11:09:28.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a16d9d700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a00005200 0x7f5a0000ba10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5a100013d0 tx=0x7f5a00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224a48e06cc45738 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10012600 con 0x7f5a00005200 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a100127a0 con 0x7f5a00005200 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1559a700 1 -- 172.21.15.83:0/255300784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10012b50 con 0x7f5a00005200 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/255300784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a00005200 msgr2=0x7f5a0000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a00005200 0x7f5a0000ba10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5a100013d0 tx=0x7f5a00005880 comp rx=0 tx=0).stop 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/255300784 shutdown_connections 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/255300784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a00005200 0x7f5a0000ba10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/255300784 >> 172.21.15.83:0/255300784 conn(0x7f5a180cc690 msgr2=0x7f5a180cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/255300784 shutdown_connections 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.470+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/255300784 wait complete. 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.472+0000 7f5a1ca9e700 1 Processor -- start 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.472+0000 7f5a1ca9e700 1 -- start start 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00005200 0x7f5a180c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 0x7f5a180d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a18152ef0 0x7f5a18153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180cea00 con 0x7f5a18150220 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a181538a0 con 0x7f5a00005200 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180cf540 con 0x7f5a18152ef0 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 0x7f5a180d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 0x7f5a180d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50136/0 (socket says 172.21.15.83:50136) 2023-11-28T11:09:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 -- 172.21.15.83:0/1229413085 learned_addr learned my addr 172.21.15.83:0/1229413085 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1659c700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00005200 0x7f5a180c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a16d9d700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a18152ef0 0x7f5a18153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 -- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a18152ef0 msgr2=0x7f5a18153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a18152ef0 0x7f5a18153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 -- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00005200 msgr2=0x7f5a180c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00005200 0x7f5a180c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 -- 172.21.15.83:0/1229413085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180cbee0 con 0x7f5a18150220 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a15d9b700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 0x7f5a180d0270 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800f670 tx=0x7f5a0800f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080052e0 con 0x7f5a18150220 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08005440 con 0x7f5a18150220 2023-11-28T11:09:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a180cbc00 con 0x7f5a18150220 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080038a0 con 0x7f5a18150220 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.473+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a180cc1c0 con 0x7f5a18150220 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.474+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a180742b0 con 0x7f5a18150220 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.475+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5a08003a00 con 0x7f5a18150220 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.475+0000 7f5a06ffd700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59f80682f0 0x7f59f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.475+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5a08085cc0 con 0x7f5a18150220 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.475+0000 7f5a1659c700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59f80682f0 0x7f59f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.475+0000 7f5a1659c700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59f80682f0 0x7f59f806a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009be0 tx=0x7f5a0c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.479+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5a080521f0 con 0x7f5a18150220 2023-11-28T11:09:28.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.662+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a180cf680 con 0x7f5a18150220 2023-11-28T11:09:28.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.663+0000 7f5a06ffd700 1 -- 172.21.15.83:0/1229413085 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a08055810 con 0x7f5a18150220 2023-11-28T11:09:28.664 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:28.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59f80682f0 msgr2=0x7f59f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59f80682f0 0x7f59f806a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009be0 tx=0x7f5a0c009210 comp rx=0 tx=0).stop 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 msgr2=0x7f5a180d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 0x7f5a180d0270 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800f670 tx=0x7f5a0800f980 comp rx=0 tx=0).stop 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 shutdown_connections 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a18152ef0 0x7f5a18153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a00005200 0x7f5a180c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f59f80682f0 0x7f59f806a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 --2- 172.21.15.83:0/1229413085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a18150220 0x7f5a180d0270 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 >> 172.21.15.83:0/1229413085 conn(0x7f5a180cc690 msgr2=0x7f5a180b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 shutdown_connections 2023-11-28T11:09:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.665+0000 7f5a1ca9e700 1 -- 172.21.15.83:0/1229413085 wait complete. 2023-11-28T11:09:28.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:28.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.799+0000 7f77f9ef1700 1 Processor -- start 2023-11-28T11:09:28.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.799+0000 7f77f9ef1700 1 -- start start 2023-11-28T11:09:28.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f9ef1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77f415c890 0x7f77f40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f9ef1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f406d4a0 con 0x7f77f406e370 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f9ef1700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40b5ae0 con 0x7f77f40b4bb0 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f9ef1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f40b5020 con 0x7f77f415c890 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77f415c890 0x7f77f40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f3fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f77f406e370 0x7f77f406b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48006/0 (socket says 172.21.15.83:48006) 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.800+0000 7f77f3fff700 1 -- 172.21.15.83:0/70057121 learned_addr learned my addr 172.21.15.83:0/70057121 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f37fe700 1 -- 172.21.15.83:0/70057121 >> v1:172.21.15.90:6789/0 conn(0x7f77f40b4bb0 legacy=0x7f77f406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f37fe700 1 -- 172.21.15.83:0/70057121 >> v1:172.21.15.83:6789/0 conn(0x7f77f406e370 legacy=0x7f77f406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f37fe700 1 -- 172.21.15.83:0/70057121 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f406c400 con 0x7f77f415c890 2023-11-28T11:09:28.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f27fc700 1 -- 172.21.15.83:0/70057121 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324700339 0 0) 0x7f77f406d4a0 con 0x7f77f406e370 2023-11-28T11:09:28.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f37fe700 1 --2- 172.21.15.83:0/70057121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f415c890 0x7f77f40aee60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009820 tx=0x7f77dc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863f2fdf23c548b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f27fc700 1 -- 172.21.15.83:0/70057121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc005e30 con 0x7f77f415c890 2023-11-28T11:09:28.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f27fc700 1 -- 172.21.15.83:0/70057121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc004030 con 0x7f77f415c890 2023-11-28T11:09:28.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f27fc700 1 -- 172.21.15.83:0/70057121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0043c0 con 0x7f77f415c890 2023-11-28T11:09:28.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f9ef1700 1 -- 172.21.15.83:0/70057121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f415c890 msgr2=0x7f77f40aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/70057121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f415c890 0x7f77f40aee60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009820 tx=0x7f77dc009b30 comp rx=0 tx=0).stop 2023-11-28T11:09:28.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f9ef1700 1 -- 172.21.15.83:0/70057121 shutdown_connections 2023-11-28T11:09:28.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/70057121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f415c890 0x7f77f40aee60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f9ef1700 1 -- 172.21.15.83:0/70057121 >> 172.21.15.83:0/70057121 conn(0x7f77f40b2b60 msgr2=0x7f77f40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.801+0000 7f77f9ef1700 1 -- 172.21.15.83:0/70057121 shutdown_connections 2023-11-28T11:09:28.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.802+0000 7f77f9ef1700 1 -- 172.21.15.83:0/70057121 wait complete. 2023-11-28T11:09:28.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.803+0000 7f77f9ef1700 1 Processor -- start 2023-11-28T11:09:28.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.803+0000 7f77f9ef1700 1 -- start start 2023-11-28T11:09:28.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.803+0000 7f77f9ef1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77f406e370 0x7f77f4151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 0x7f77f4158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77f415c890 0x7f77f4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40c7150 con 0x7f77f40b4bb0 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40c7430 con 0x7f77f415c890 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40c7710 con 0x7f77f406e370 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 0x7f77f4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 0x7f77f4158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50150/0 (socket says 172.21.15.83:50150) 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 -- 172.21.15.83:0/2915163406 learned_addr learned my addr 172.21.15.83:0/2915163406 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f3fff700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77f415c890 0x7f77f4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f37fe700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77f406e370 0x7f77f4151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 -- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77f406e370 msgr2=0x7f77f4151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77f406e370 0x7f77f4151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 -- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77f415c890 msgr2=0x7f77f4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77f415c890 0x7f77f4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 -- 172.21.15.83:0/2915163406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc009430 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f2ffd700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 0x7f77f4158700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77e400b4a0 tx=0x7f77e400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4011830 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4011990 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4011d40 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f40b5520 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.804+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f40b5240 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.805+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f40b5800 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.806+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f77e400f050 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.806+0000 7f77ebfff700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f77d40682f0 0x7f77d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.806+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f77e4085bb0 con 0x7f77f40b4bb0 2023-11-28T11:09:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.806+0000 7f77f37fe700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f77d40682f0 0x7f77d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.807+0000 7f77f37fe700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f77d40682f0 0x7f77d406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77dc000c00 tx=0x7f77dc008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.810+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f77e408c080 con 0x7f77f40b4bb0 2023-11-28T11:09:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.992+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77f406ac20 con 0x7f77f40b4bb0 2023-11-28T11:09:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.992+0000 7f77ebfff700 1 -- 172.21.15.83:0/2915163406 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77e40150a0 con 0x7f77f40b4bb0 2023-11-28T11:09:28.994 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f77d40682f0 msgr2=0x7f77d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f77d40682f0 0x7f77d406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77dc000c00 tx=0x7f77dc008070 comp rx=0 tx=0).stop 2023-11-28T11:09:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 msgr2=0x7f77f4158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 0x7f77f4158700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77e400b4a0 tx=0x7f77e400b860 comp rx=0 tx=0).stop 2023-11-28T11:09:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 shutdown_connections 2023-11-28T11:09:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77f415c890 0x7f77f4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77f406e370 0x7f77f4151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f77d40682f0 0x7f77d406a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 --2- 172.21.15.83:0/2915163406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77f40b4bb0 0x7f77f4158700 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.994+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 >> 172.21.15.83:0/2915163406 conn(0x7f77f40b2b60 msgr2=0x7f77f415b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.995+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 shutdown_connections 2023-11-28T11:09:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:28.995+0000 7f77f9ef1700 1 -- 172.21.15.83:0/2915163406 wait complete. 2023-11-28T11:09:29.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:29.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.129+0000 7fa46bb07700 1 Processor -- start 2023-11-28T11:09:29.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.130+0000 7fa46bb07700 1 -- start start 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.130+0000 7fa46bb07700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa4640d0560 0x7fa4640ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.130+0000 7fa46bb07700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 0x7fa4640b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.130+0000 7fa46bb07700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4640b3660 con 0x7fa464158040 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.130+0000 7fa46bb07700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640cea60 con 0x7fa4640d0560 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46bb07700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4640d0a00 con 0x7fa4640cd4f0 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 0x7fa4640b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 0x7fa4640b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:34362/0 (socket says 172.21.15.83:34362) 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa4690a2700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa4640d0560 0x7fa4640ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 -- 172.21.15.83:0/3466196159 learned_addr learned my addr 172.21.15.83:0/3466196159 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 -- 172.21.15.83:0/3466196159 >> v2:172.21.15.90:3300/0 conn(0x7fa4640d0560 msgr2=0x7fa4640ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 --2- 172.21.15.83:0/3466196159 >> v2:172.21.15.90:3300/0 conn(0x7fa4640d0560 0x7fa4640ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 -- 172.21.15.83:0/3466196159 >> v1:172.21.15.90:6789/0 conn(0x7fa464158040 legacy=0x7fa46415b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 -- 172.21.15.83:0/3466196159 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640cb450 con 0x7fa4640cd4f0 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa4690a2700 1 --2- 172.21.15.83:0/3466196159 >> v2:172.21.15.90:3300/0 conn(0x7fa4640d0560 0x7fa4640ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa46a0a4700 1 --2- 172.21.15.83:0/3466196159 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 0x7fa4640b27e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4540098f0 tx=0x7fa454009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8915087b636dfe5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.131+0000 7fa4688a1700 1 -- 172.21.15.83:0/3466196159 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454011830 con 0x7fa4640cd4f0 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa4688a1700 1 -- 172.21.15.83:0/3466196159 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa454011990 con 0x7fa4640cd4f0 2023-11-28T11:09:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa4688a1700 1 -- 172.21.15.83:0/3466196159 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454011d20 con 0x7fa4640cd4f0 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 -- 172.21.15.83:0/3466196159 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 msgr2=0x7fa4640b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 --2- 172.21.15.83:0/3466196159 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 0x7fa4640b27e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4540098f0 tx=0x7fa454009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 -- 172.21.15.83:0/3466196159 shutdown_connections 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 --2- 172.21.15.83:0/3466196159 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640cd4f0 0x7fa4640b27e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 --2- 172.21.15.83:0/3466196159 >> v2:172.21.15.90:3300/0 conn(0x7fa4640d0560 0x7fa4640ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 -- 172.21.15.83:0/3466196159 >> 172.21.15.83:0/3466196159 conn(0x7fa46406de20 msgr2=0x7fa46406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 -- 172.21.15.83:0/3466196159 shutdown_connections 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.132+0000 7fa46bb07700 1 -- 172.21.15.83:0/3466196159 wait complete. 2023-11-28T11:09:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.133+0000 7fa46bb07700 1 Processor -- start 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.133+0000 7fa46bb07700 1 -- start start 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46bb07700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4640cd4f0 0x7fa4640b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46bb07700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640d0560 0x7fa464155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46bb07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 0x7fa4641592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46bb07700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4641e4f60 con 0x7fa464158040 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46bb07700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4640b4ff0 con 0x7fa4640cd4f0 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46bb07700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4640b40d0 con 0x7fa4640d0560 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 0x7fa4641592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 0x7fa4641592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50166/0 (socket says 172.21.15.83:50166) 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 -- 172.21.15.83:0/2005550257 learned_addr learned my addr 172.21.15.83:0/2005550257 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa4690a2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640d0560 0x7fa464155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa4698a3700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4640cd4f0 0x7fa4640b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 -- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640d0560 msgr2=0x7fa464155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640d0560 0x7fa464155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 -- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4640cd4f0 msgr2=0x7fa4640b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4640cd4f0 0x7fa4640b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.134+0000 7fa46a0a4700 1 -- 172.21.15.83:0/2005550257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460009a70 con 0x7fa464158040 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.135+0000 7fa46a0a4700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 0x7fa4641592d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa454013000 tx=0x7fa454004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.135+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454004380 con 0x7fa464158040 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.135+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45401f070 con 0x7fa464158040 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.135+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa454009550 con 0x7fa464158040 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.135+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4540036a0 con 0x7fa464158040 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.135+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa464158410 con 0x7fa464158040 2023-11-28T11:09:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.136+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4640cb730 con 0x7fa464158040 2023-11-28T11:09:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.136+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa45402a020 con 0x7fa464158040 2023-11-28T11:09:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.136+0000 7fa45a7fc700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa4480682f0 0x7fa44806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.136+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa4540874d0 con 0x7fa464158040 2023-11-28T11:09:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.137+0000 7fa4698a3700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa4480682f0 0x7fa44806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.137+0000 7fa4698a3700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa4480682f0 0x7fa44806a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa464157860 tx=0x7fa45c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.141+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa454053980 con 0x7fa464158040 2023-11-28T11:09:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.322+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa4640cba10 con 0x7fa464158040 2023-11-28T11:09:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.322+0000 7fa45a7fc700 1 -- 172.21.15.83:0/2005550257 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa454056fa0 con 0x7fa464158040 2023-11-28T11:09:29.324 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa4480682f0 msgr2=0x7fa44806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa4480682f0 0x7fa44806a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa464157860 tx=0x7fa45c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 msgr2=0x7fa4641592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 0x7fa4641592d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa454013000 tx=0x7fa454004750 comp rx=0 tx=0).stop 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 shutdown_connections 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa4480682f0 0x7fa44806a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa464158040 0x7fa4641592d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4640cd4f0 0x7fa4640b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 --2- 172.21.15.83:0/2005550257 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa4640d0560 0x7fa464155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 >> 172.21.15.83:0/2005550257 conn(0x7fa46406de20 msgr2=0x7fa464159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 shutdown_connections 2023-11-28T11:09:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.324+0000 7fa46bb07700 1 -- 172.21.15.83:0/2005550257 wait complete. 2023-11-28T11:09:29.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.457+0000 7f64eee7b700 1 Processor -- start 2023-11-28T11:09:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.458+0000 7f64eee7b700 1 -- start start 2023-11-28T11:09:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.458+0000 7f64eee7b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e806e370 0x7f64e80aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.458+0000 7f64eee7b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e806d4a0 con 0x7f64e80b4bb0 2023-11-28T11:09:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.458+0000 7f64eee7b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e80b5ae0 con 0x7f64e815c890 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.458+0000 7f64eee7b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80b5050 con 0x7f64e806e370 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64ed418700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f64e80b4bb0 0x7f64e806b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48034/0 (socket says 172.21.15.83:48034) 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64ed418700 1 -- 172.21.15.83:0/1167080310 learned_addr learned my addr 172.21.15.83:0/1167080310 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557314521 0 0) 0x7f64e806d4a0 con 0x7f64e80b4bb0 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e7fff700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e806e370 0x7f64e80aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d80035b0 con 0x7f64e80b4bb0 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2376544470 0 0) 0x7f64d80035b0 con 0x7f64e80b4bb0 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e806e370 msgr2=0x7f64e80aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e806e370 0x7f64e80aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 >> v1:172.21.15.90:6789/0 conn(0x7f64e815c890 legacy=0x7f64e806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e806c400 con 0x7f64e80b4bb0 2023-11-28T11:09:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f64e00019f0 con 0x7f64e80b4bb0 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 >> v1:172.21.15.83:6789/0 conn(0x7f64e80b4bb0 legacy=0x7f64e806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64d8003a30 0x7f64d8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64d8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64d8005250 0x7f64d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d800c110 con 0x7f64d8004850 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d800c280 con 0x7f64d8005250 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d800c440 con 0x7f64d8003a30 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e7fff700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e806e370 0x7f64e80aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64ecc17700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64d8003a30 0x7f64d8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64ed418700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64d8005250 0x7f64d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.459+0000 7f64e7fff700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64d8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e7fff700 1 -- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64d8003a30 msgr2=0x7f64d8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e7fff700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64d8003a30 0x7f64d8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e7fff700 1 -- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64d8005250 msgr2=0x7f64d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e7fff700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64d8005250 0x7f64d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e7fff700 1 -- 172.21.15.83:0/1167080310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80035b0 con 0x7f64d8004850 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64ecc17700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64d8003a30 0x7f64d8003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e7fff700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64d8004cc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64e80d0c90 tx=0x7f64d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad6cf508e3078ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400c070 con 0x7f64d8004850 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d4013630 con 0x7f64d8004850 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64e77fe700 1 -- 172.21.15.83:0/1167080310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40093d0 con 0x7f64d8004850 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64eee7b700 1 -- 172.21.15.83:0/1167080310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 msgr2=0x7f64d8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64eee7b700 1 --2- 172.21.15.83:0/1167080310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64d8004cc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64e80d0c90 tx=0x7f64d8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64ed418700 1 -- 172.21.15.83:0/1167080310 reap_dead start 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64eee7b700 1 -- 172.21.15.83:0/1167080310 shutdown_connections 2023-11-28T11:09:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.460+0000 7f64eee7b700 1 -- 172.21.15.83:0/1167080310 >> 172.21.15.83:0/1167080310 conn(0x7f64e80b2b60 msgr2=0x7f64e80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.461+0000 7f64eee7b700 1 -- 172.21.15.83:0/1167080310 shutdown_connections 2023-11-28T11:09:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.461+0000 7f64eee7b700 1 -- 172.21.15.83:0/1167080310 wait complete. 2023-11-28T11:09:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.462+0000 7f64eee7b700 1 Processor -- start 2023-11-28T11:09:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.462+0000 7f64eee7b700 1 -- start start 2023-11-28T11:09:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64e81f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64e815dcb0 0x7f64e815e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e815e660 0x7f64e80b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80ca5a0 con 0x7f64d8004850 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e815ead0 con 0x7f64e815dcb0 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80c9f20 con 0x7f64e815e660 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64e81f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64e81f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50182/0 (socket says 172.21.15.83:50182) 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 -- 172.21.15.83:0/2429657258 learned_addr learned my addr 172.21.15.83:0/2429657258 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ed418700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e815e660 0x7f64e80b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 -- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e815e660 msgr2=0x7f64e80b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64e7fff700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64e815dcb0 0x7f64e815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e815e660 0x7f64e80b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 -- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64e815dcb0 msgr2=0x7f64e815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64e815dcb0 0x7f64e815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 -- 172.21.15.83:0/2429657258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80b5520 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64e7fff700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64e815dcb0 0x7f64e815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64ecc17700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64e81f19d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00f840 tx=0x7f64dc00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0052e0 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc005440 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e80b5240 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0055b0 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.463+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e80b5800 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.464+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e80742b0 con 0x7f64d8004850 2023-11-28T11:09:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.465+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f64dc0038f0 con 0x7f64d8004850 2023-11-28T11:09:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.465+0000 7f64e57fa700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64cc0682f0 0x7f64cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.465+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f64dc0860a0 con 0x7f64d8004850 2023-11-28T11:09:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.465+0000 7f64e7fff700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64cc0682f0 0x7f64cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.465+0000 7f64e7fff700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64cc0682f0 0x7f64cc06a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64e815fc60 tx=0x7f64d400c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.469+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f64dc0525d0 con 0x7f64d8004850 2023-11-28T11:09:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.667+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64e80b3820 con 0x7f64d8004850 2023-11-28T11:09:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.668+0000 7f64e57fa700 1 -- 172.21.15.83:0/2429657258 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f64dc055bf0 con 0x7f64d8004850 2023-11-28T11:09:29.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64cc0682f0 msgr2=0x7f64cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64cc0682f0 0x7f64cc06a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64e815fc60 tx=0x7f64d400c000 comp rx=0 tx=0).stop 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 msgr2=0x7f64e81f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64e81f19d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00f840 tx=0x7f64dc00fc00 comp rx=0 tx=0).stop 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 shutdown_connections 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64e815e660 0x7f64e80b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64cc0682f0 0x7f64cc06a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64d8004850 0x7f64e81f19d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 --2- 172.21.15.83:0/2429657258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64e815dcb0 0x7f64e815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 >> 172.21.15.83:0/2429657258 conn(0x7f64e80b2b60 msgr2=0x7f64e80ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 shutdown_connections 2023-11-28T11:09:29.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.670+0000 7f64eee7b700 1 -- 172.21.15.83:0/2429657258 wait complete. 2023-11-28T11:09:29.673 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:09:29.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T11:09:29.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:29.683 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T11:09:29.726 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:09:29.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:29.727 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:09:29.786 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:09:29.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.906+0000 7fca03197700 1 Processor -- start 2023-11-28T11:09:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.906+0000 7fca03197700 1 -- start start 2023-11-28T11:09:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.906+0000 7fca03197700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9fc15a3f0 0x7fc9fc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.906+0000 7fca03197700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc0cb1d0 con 0x7fc9fc15cff0 2023-11-28T11:09:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.906+0000 7fca03197700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc0b2030 con 0x7fc9fc0b1720 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.906+0000 7fca03197700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9fc0cb540 con 0x7fc9fc15a3f0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9fc15a3f0 0x7fc9fc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fca01734700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc9fc15cff0 0x7fc9fc0cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48054/0 (socket says 172.21.15.83:48054) 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fca01734700 1 -- 172.21.15.83:0/3831951505 learned_addr learned my addr 172.21.15.83:0/3831951505 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fbfff700 1 -- 172.21.15.83:0/3831951505 >> v1:172.21.15.90:6789/0 conn(0x7fc9fc0b1720 legacy=0x7fc9fc0cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fbfff700 1 -- 172.21.15.83:0/3831951505 >> v1:172.21.15.83:6789/0 conn(0x7fc9fc15cff0 legacy=0x7fc9fc0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fb7fe700 1 -- 172.21.15.83:0/3831951505 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245153422 0 0) 0x7fc9fc0cb1d0 con 0x7fc9fc15cff0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fbfff700 1 -- 172.21.15.83:0/3831951505 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc15d840 con 0x7fc9fc15a3f0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fbfff700 1 --2- 172.21.15.83:0/3831951505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc0b1af0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009840 tx=0x7fc9e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad06dcca86fa30b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fb7fe700 1 -- 172.21.15.83:0/3831951505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8011830 con 0x7fc9fc15a3f0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fb7fe700 1 -- 172.21.15.83:0/3831951505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e8011990 con 0x7fc9fc15a3f0 2023-11-28T11:09:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.907+0000 7fc9fb7fe700 1 -- 172.21.15.83:0/3831951505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8011d20 con 0x7fc9fc15a3f0 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 -- 172.21.15.83:0/3831951505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc15a3f0 msgr2=0x7fc9fc0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 --2- 172.21.15.83:0/3831951505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc0b1af0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009840 tx=0x7fc9e8009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 -- 172.21.15.83:0/3831951505 shutdown_connections 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 --2- 172.21.15.83:0/3831951505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc0b1af0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 -- 172.21.15.83:0/3831951505 >> 172.21.15.83:0/3831951505 conn(0x7fc9fc0cc720 msgr2=0x7fc9fc0ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 -- 172.21.15.83:0/3831951505 shutdown_connections 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.908+0000 7fca03197700 1 -- 172.21.15.83:0/3831951505 wait complete. 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.909+0000 7fca03197700 1 Processor -- start 2023-11-28T11:09:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.909+0000 7fca03197700 1 -- start start 2023-11-28T11:09:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca03197700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 0x7fc9fc15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca03197700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca03197700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc9fc15cff0 0x7fc9fc147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca03197700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc147e80 con 0x7fc9fc0b1720 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca03197700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0c77d0 con 0x7fc9fc15a3f0 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca03197700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc1483b0 con 0x7fc9fc15cff0 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca00f33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 0x7fc9fc15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca00f33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 0x7fc9fc15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50198/0 (socket says 172.21.15.83:50198) 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.910+0000 7fca00f33700 1 -- 172.21.15.83:0/2704989590 learned_addr learned my addr 172.21.15.83:0/2704989590 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fc9fbfff700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca01734700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc9fc15cff0 0x7fc9fc147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca00f33700 1 -- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc9fc15cff0 msgr2=0x7fc9fc147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca00f33700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc9fc15cff0 0x7fc9fc147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca00f33700 1 -- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9fc15a3f0 msgr2=0x7fc9fc147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca00f33700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca00f33700 1 -- 172.21.15.83:0/2704989590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80094a0 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fc9fbfff700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca00f33700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 0x7fc9fc15c290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000c4e0 tx=0x7fc9f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00052d0 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f0005430 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00038c0 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc15de00 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.911+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc15db20 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.912+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc15a870 con 0x7fc9fc0b1720 2023-11-28T11:09:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.912+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9f0019030 con 0x7fc9fc0b1720 2023-11-28T11:09:29.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.913+0000 7fc9f97fa700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc9e00682f0 0x7fc9e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.913+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc9f0085330 con 0x7fc9fc0b1720 2023-11-28T11:09:29.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.913+0000 7fc9fbfff700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc9e00682f0 0x7fc9e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.913+0000 7fc9fbfff700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc9e00682f0 0x7fc9e006a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc14e560 tx=0x7fc9e801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:29.917+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9f0051860 con 0x7fc9fc0b1720 2023-11-28T11:09:30.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.150+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9fc15d430 con 0x7fc9fc0b1720 2023-11-28T11:09:30.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.150+0000 7fc9f97fa700 1 -- 172.21.15.83:0/2704989590 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fc9f0054e80 con 0x7fc9fc0b1720 2023-11-28T11:09:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:30.152 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:51.826077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:30.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc9e00682f0 msgr2=0x7fc9e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc9e00682f0 0x7fc9e006a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc14e560 tx=0x7fc9e801c040 comp rx=0 tx=0).stop 2023-11-28T11:09:30.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 msgr2=0x7fc9fc15c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 0x7fc9fc15c290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000c4e0 tx=0x7fc9f000c8a0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 shutdown_connections 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc9fc15cff0 0x7fc9fc147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc9e00682f0 0x7fc9e006a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9fc0b1720 0x7fc9fc15c290 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 --2- 172.21.15.83:0/2704989590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9fc15a3f0 0x7fc9fc147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 >> 172.21.15.83:0/2704989590 conn(0x7fc9fc0cc720 msgr2=0x7fc9fc14e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 shutdown_connections 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.153+0000 7fca03197700 1 -- 172.21.15.83:0/2704989590 wait complete. 2023-11-28T11:09:30.155 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2023-11-28T11:09:30.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:30.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.285+0000 7fa83a705700 1 Processor -- start 2023-11-28T11:09:30.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.285+0000 7fa83a705700 1 -- start start 2023-11-28T11:09:30.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa83a705700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa83415a8e0 0x7fa8340b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa83a705700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8340b2ea0 0x7fa8340b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa83a705700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340b32d0 con 0x7fa83415a510 2023-11-28T11:09:30.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa83a705700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340cd780 con 0x7fa8340b2ea0 2023-11-28T11:09:30.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa83a705700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83415f1d0 con 0x7fa83415a8e0 2023-11-28T11:09:30.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa838ca2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8340b2ea0 0x7fa8340b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa838ca2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8340b2ea0 0x7fa8340b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50208/0 (socket says 172.21.15.83:50208) 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.286+0000 7fa838ca2700 1 -- 172.21.15.83:0/786952784 learned_addr learned my addr 172.21.15.83:0/786952784 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa8337fe700 1 --2- 172.21.15.83:0/786952784 >> v2:172.21.15.90:3300/0 conn(0x7fa83415a8e0 0x7fa8340b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa838ca2700 1 -- 172.21.15.83:0/786952784 >> v2:172.21.15.90:3300/0 conn(0x7fa83415a8e0 msgr2=0x7fa8340b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa838ca2700 1 --2- 172.21.15.83:0/786952784 >> v2:172.21.15.90:3300/0 conn(0x7fa83415a8e0 0x7fa8340b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa838ca2700 1 -- 172.21.15.83:0/786952784 >> v1:172.21.15.83:6789/0 conn(0x7fa83415a510 legacy=0x7fa8340cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa832ffd700 1 -- 172.21.15.83:0/786952784 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791054344 0 0) 0x7fa8340b32d0 con 0x7fa83415a510 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa838ca2700 1 -- 172.21.15.83:0/786952784 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83415eef0 con 0x7fa8340b2ea0 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa838ca2700 1 --2- 172.21.15.83:0/786952784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b3d30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa83404e880 tx=0x7fa82c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1521427020d2f79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa832ffd700 1 -- 172.21.15.83:0/786952784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00e070 con 0x7fa8340b2ea0 2023-11-28T11:09:30.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa832ffd700 1 -- 172.21.15.83:0/786952784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c00b9c0 con 0x7fa8340b2ea0 2023-11-28T11:09:30.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa832ffd700 1 -- 172.21.15.83:0/786952784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00bd70 con 0x7fa8340b2ea0 2023-11-28T11:09:30.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 -- 172.21.15.83:0/786952784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8340b2ea0 msgr2=0x7fa8340b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 --2- 172.21.15.83:0/786952784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b3d30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa83404e880 tx=0x7fa82c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:30.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 -- 172.21.15.83:0/786952784 shutdown_connections 2023-11-28T11:09:30.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 --2- 172.21.15.83:0/786952784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b3d30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 --2- 172.21.15.83:0/786952784 >> v2:172.21.15.90:3300/0 conn(0x7fa83415a8e0 0x7fa8340b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 -- 172.21.15.83:0/786952784 >> 172.21.15.83:0/786952784 conn(0x7fa8340cd960 msgr2=0x7fa8340cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.287+0000 7fa83a705700 1 -- 172.21.15.83:0/786952784 shutdown_connections 2023-11-28T11:09:30.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.288+0000 7fa83a705700 1 -- 172.21.15.83:0/786952784 wait complete. 2023-11-28T11:09:30.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.289+0000 7fa83a705700 1 Processor -- start 2023-11-28T11:09:30.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.289+0000 7fa83a705700 1 -- start start 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.289+0000 7fa83a705700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.289+0000 7fa83a705700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 0x7fa8340c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa83a705700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa83415a8e0 0x7fa8340b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa83a705700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c5e50 con 0x7fa83415a510 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa83a705700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c6130 con 0x7fa8340b2ea0 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa83a705700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c6410 con 0x7fa83415a8e0 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 0x7fa8340c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 0x7fa8340c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50212/0 (socket says 172.21.15.83:50212) 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 -- 172.21.15.83:0/2852124324 learned_addr learned my addr 172.21.15.83:0/2852124324 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa833fff700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa838ca2700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa83415a8e0 0x7fa8340b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 -- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa83415a8e0 msgr2=0x7fa8340b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa83415a8e0 0x7fa8340b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 -- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8340b2ea0 msgr2=0x7fa8340b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 -- 172.21.15.83:0/2852124324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009b30 con 0x7fa83415a510 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa833fff700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:30.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa8337fe700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 0x7fa8340c5420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d800 tx=0x7fa82800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280125f0 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828012750 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8340cf0b0 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828012b00 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.290+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8340cedd0 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.291+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8340cf390 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.292+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa828012c60 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.292+0000 7fa830ff9700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa820068320 0x7fa82006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.292+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa8280864c0 con 0x7fa83415a510 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.292+0000 7fa833fff700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa820068320 0x7fa82006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.292+0000 7fa833fff700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa820068320 0x7fa82006a7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa824000f60 tx=0x7fa824009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.296+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa828052970 con 0x7fa83415a510 2023-11-28T11:09:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.480+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa834074320 con 0x7fa83415a510 2023-11-28T11:09:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.482+0000 7fa830ff9700 1 -- 172.21.15.83:0/2852124324 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19737 (secure 0 0 0) 0x7fa828055f90 con 0x7fa83415a510 2023-11-28T11:09:30.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:30.483 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:25.004035+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:30.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa820068320 msgr2=0x7fa82006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa820068320 0x7fa82006a7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa824000f60 tx=0x7fa824009040 comp rx=0 tx=0).stop 2023-11-28T11:09:30.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 msgr2=0x7fa8340c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 0x7fa8340c5420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d800 tx=0x7fa82800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 shutdown_connections 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa83415a8e0 0x7fa8340b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8340b2ea0 0x7fa8340b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa820068320 0x7fa82006a7d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 --2- 172.21.15.83:0/2852124324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa83415a510 0x7fa8340c5420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 >> 172.21.15.83:0/2852124324 conn(0x7fa8340cd960 msgr2=0x7fa8340b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 shutdown_connections 2023-11-28T11:09:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.484+0000 7fa83a705700 1 -- 172.21.15.83:0/2852124324 wait complete. 2023-11-28T11:09:30.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:33.960379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:34.471584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:37.555175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:30.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.619+0000 7fa1c0323700 1 Processor -- start 2023-11-28T11:09:30.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.619+0000 7fa1c0323700 1 -- start start 2023-11-28T11:09:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1c0323700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806c500 0x7fa1b806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1c0323700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa1b806f440 0x7fa1b80ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1c0323700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80cd6a0 con 0x7fa1b80af040 2023-11-28T11:09:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1c0323700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80cce00 con 0x7fa1b806f440 2023-11-28T11:09:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1c0323700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80cd0e0 con 0x7fa1b806c500 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1bd8be700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806c500 0x7fa1b806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1be8c0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa1b806f440 0x7fa1b80ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1be0bf700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fa1b80af040 0x7fa1b80cc5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:46424/0 (socket says 172.21.15.83:46424) 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.620+0000 7fa1be0bf700 1 -- 172.21.15.83:0/3927153657 learned_addr learned my addr 172.21.15.83:0/3927153657 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd8be700 1 -- 172.21.15.83:0/3927153657 >> v2:172.21.15.90:3300/0 conn(0x7fa1b806f440 msgr2=0x7fa1b80ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd8be700 1 --2- 172.21.15.83:0/3927153657 >> v2:172.21.15.90:3300/0 conn(0x7fa1b806f440 0x7fa1b80ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd8be700 1 -- 172.21.15.83:0/3927153657 >> v1:172.21.15.90:6789/0 conn(0x7fa1b80af040 legacy=0x7fa1b80cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd8be700 1 -- 172.21.15.83:0/3927153657 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80cd3c0 con 0x7fa1b806c500 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1be8c0700 1 --2- 172.21.15.83:0/3927153657 >> v2:172.21.15.90:3300/0 conn(0x7fa1b806f440 0x7fa1b80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd8be700 1 --2- 172.21.15.83:0/3927153657 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806c500 0x7fa1b806ef00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009840 tx=0x7fa1b4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5007daab275ed59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd0bd700 1 -- 172.21.15.83:0/3927153657 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4011830 con 0x7fa1b806c500 2023-11-28T11:09:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd0bd700 1 -- 172.21.15.83:0/3927153657 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b4011990 con 0x7fa1b806c500 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.621+0000 7fa1bd0bd700 1 -- 172.21.15.83:0/3927153657 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4011d20 con 0x7fa1b806c500 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 -- 172.21.15.83:0/3927153657 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806c500 msgr2=0x7fa1b806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 --2- 172.21.15.83:0/3927153657 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806c500 0x7fa1b806ef00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009840 tx=0x7fa1b4009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 -- 172.21.15.83:0/3927153657 shutdown_connections 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 --2- 172.21.15.83:0/3927153657 >> v2:172.21.15.90:3300/0 conn(0x7fa1b806f440 0x7fa1b80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 --2- 172.21.15.83:0/3927153657 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806c500 0x7fa1b806ef00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 -- 172.21.15.83:0/3927153657 >> 172.21.15.83:0/3927153657 conn(0x7fa1b80b2c10 msgr2=0x7fa1b80b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 -- 172.21.15.83:0/3927153657 shutdown_connections 2023-11-28T11:09:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.622+0000 7fa1c0323700 1 -- 172.21.15.83:0/3927153657 wait complete. 2023-11-28T11:09:30.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.623+0000 7fa1c0323700 1 Processor -- start 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.623+0000 7fa1c0323700 1 -- start start 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1c0323700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 0x7fa1b81461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1c0323700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806f440 0x7fa1b8157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1c0323700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1b80af040 0x7fa1b80c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1c0323700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c9af0 con 0x7fa1b806c500 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1c0323700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c7ce0 con 0x7fa1b80af040 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1c0323700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c7f60 con 0x7fa1b806f440 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1be0bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 0x7fa1b81461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1be0bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 0x7fa1b81461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50224/0 (socket says 172.21.15.83:50224) 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.624+0000 7fa1be0bf700 1 -- 172.21.15.83:0/2185388358 learned_addr learned my addr 172.21.15.83:0/2185388358 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1bd8be700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806f440 0x7fa1b8157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be8c0700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1b80af040 0x7fa1b80c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be0bf700 1 -- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806f440 msgr2=0x7fa1b8157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be0bf700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806f440 0x7fa1b8157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be0bf700 1 -- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1b80af040 msgr2=0x7fa1b80c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be0bf700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1b80af040 0x7fa1b80c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be0bf700 1 -- 172.21.15.83:0/2185388358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0009a70 con 0x7fa1b806c500 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1bd8be700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806f440 0x7fa1b8157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1be0bf700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 0x7fa1b81461c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e940 tx=0x7fa1a800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8005d70 con 0x7fa1b806c500 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a800ee20 con 0x7fa1b806c500 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b40094a0 con 0x7fa1b806c500 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8001810 con 0x7fa1b806c500 2023-11-28T11:09:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.625+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b806f890 con 0x7fa1b806c500 2023-11-28T11:09:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.626+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b80ca420 con 0x7fa1b806c500 2023-11-28T11:09:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.626+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa1a8021070 con 0x7fa1b806c500 2023-11-28T11:09:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.627+0000 7fa1aeffd700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa19c0682f0 0x7fa19c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.627+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa1a80880f0 con 0x7fa1b806c500 2023-11-28T11:09:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.627+0000 7fa1bd8be700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa19c0682f0 0x7fa19c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.627+0000 7fa1bd8be700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa19c0682f0 0x7fa19c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80cb4d0 tx=0x7fa1b4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.631+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa1a80546e0 con 0x7fa1b806c500 2023-11-28T11:09:30.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.808+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1b80af410 con 0x7fa1b806c500 2023-11-28T11:09:30.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.809+0000 7fa1aeffd700 1 -- 172.21.15.83:0/2185388358 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1a8057d00 con 0x7fa1b806c500 2023-11-28T11:09:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:30.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa19c0682f0 msgr2=0x7fa19c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa19c0682f0 0x7fa19c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80cb4d0 tx=0x7fa1b4008030 comp rx=0 tx=0).stop 2023-11-28T11:09:30.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 msgr2=0x7fa1b81461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 0x7fa1b81461c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e940 tx=0x7fa1a800ed00 comp rx=0 tx=0).stop 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 shutdown_connections 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1b80af040 0x7fa1b80c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa19c0682f0 0x7fa19c06a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b806c500 0x7fa1b81461c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 --2- 172.21.15.83:0/2185388358 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa1b806f440 0x7fa1b8157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 >> 172.21.15.83:0/2185388358 conn(0x7fa1b80b2c10 msgr2=0x7fa1b80cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 shutdown_connections 2023-11-28T11:09:30.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.811+0000 7fa1c0323700 1 -- 172.21.15.83:0/2185388358 wait complete. 2023-11-28T11:09:30.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:30.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.945+0000 7f6ecc2f6700 1 Processor -- start 2023-11-28T11:09:30.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.945+0000 7f6ecc2f6700 1 -- start start 2023-11-28T11:09:30.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ecc2f6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 0x7f6ec415e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ecc2f6700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6ec415b3b0 0x7f6ec40d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ecc2f6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec4151a30 con 0x7f6ec415b820 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ecc2f6700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40cb460 con 0x7f6ec415b3b0 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ecc2f6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40cb740 con 0x7f6ec415e5c0 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6eca893700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ec415b820 0x7f6ec40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48094/0 (socket says 172.21.15.83:48094) 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6eca893700 1 -- 172.21.15.83:0/3725723558 learned_addr learned my addr 172.21.15.83:0/3725723558 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ec9891700 1 --2- 172.21.15.83:0/3725723558 >> v2:172.21.15.90:3300/0 conn(0x7f6ec415b3b0 0x7f6ec40d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6eca092700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 0x7f6ec415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986883718 0 0) 0x7f6ec4151a30 con 0x7f6ec415b820 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.946+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb00035b0 con 0x7f6ec415b820 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764294034 0 0) 0x7f6eb00035b0 con 0x7f6ec415b820 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 msgr2=0x7f6ec415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 0x7f6ec415e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 >> v2:172.21.15.90:3300/0 conn(0x7f6ec415b3b0 msgr2=0x7f6ec40d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 --2- 172.21.15.83:0/3725723558 >> v2:172.21.15.90:3300/0 conn(0x7f6ec415b3b0 0x7f6ec40d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec415abb0 con 0x7f6ec415b820 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6ebc0019f0 con 0x7f6ec415b820 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 >> v1:172.21.15.83:6789/0 conn(0x7f6ec415b820 legacy=0x7f6ec40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6eb0003a00 0x7f6eb0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 0x7f6eb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6eb0004d90 0x7f6eb0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40cb460 con 0x7f6eb0003a00 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40cb740 con 0x7f6eb0004d90 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb000ba10 con 0x7f6ec415e5c0 2023-11-28T11:09:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6eb0003a00 0x7f6eb0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9891700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 0x7f6eb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca893700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6eb0004d90 0x7f6eb0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 -- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 msgr2=0x7f6eb0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec415e5c0 0x7f6eb0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 -- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6eb0004d90 msgr2=0x7f6eb0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6eb0004d90 0x7f6eb0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 -- 172.21.15.83:0/3725723558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb00035b0 con 0x7f6eb0003a00 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca893700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6eb0004d90 0x7f6eb0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6eca092700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6eb0003a00 0x7f6eb0003e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009b60 tx=0x7f6eb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ca87c13206bb8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400e6c0 con 0x7f6eb0003a00 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb400e820 con 0x7f6eb0003a00 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.947+0000 7f6ec9090700 1 -- 172.21.15.83:0/3725723558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400eb90 con 0x7f6eb0003a00 2023-11-28T11:09:30.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/3725723558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6eb0003a00 msgr2=0x7f6eb0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/3725723558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6eb0003a00 0x7f6eb0003e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009b60 tx=0x7f6eb0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6eca893700 1 -- 172.21.15.83:0/3725723558 reap_dead start 2023-11-28T11:09:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/3725723558 shutdown_connections 2023-11-28T11:09:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/3725723558 >> 172.21.15.83:0/3725723558 conn(0x7f6ec406ea50 msgr2=0x7f6ec4070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/3725723558 shutdown_connections 2023-11-28T11:09:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.948+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/3725723558 wait complete. 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.950+0000 7f6ecc2f6700 1 Processor -- start 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.950+0000 7f6ecc2f6700 1 -- start start 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ec41e62f0 0x7f6ec41e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec41e3df0 0x7f6ec41503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 0x7f6ec4150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41e4350 con 0x7f6ec41e66c0 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41e4630 con 0x7f6ec41e62f0 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41e4910 con 0x7f6ec41e3df0 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 0x7f6ec4150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 0x7f6ec4150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50246/0 (socket says 172.21.15.83:50246) 2023-11-28T11:09:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 -- 172.21.15.83:0/1241390360 learned_addr learned my addr 172.21.15.83:0/1241390360 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca092700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ec41e62f0 0x7f6ec41e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ec9891700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec41e3df0 0x7f6ec41503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 -- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec41e3df0 msgr2=0x7f6ec41503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec41e3df0 0x7f6ec41503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 -- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ec41e62f0 msgr2=0x7f6ec41e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ec41e62f0 0x7f6ec41e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 -- 172.21.15.83:0/1241390360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40cba20 con 0x7f6ec41e66c0 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ec9891700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec41e3df0 0x7f6ec41503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6eca893700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 0x7f6ec4150900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ec406f640 tx=0x7f6ebc00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00ec70 con 0x7f6ec41e66c0 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc00edf0 con 0x7f6ec41e66c0 2023-11-28T11:09:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc013660 con 0x7f6ec41e66c0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec415ae90 con 0x7f6ec41e66c0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.951+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec40cbbb0 con 0x7f6ec41e66c0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.952+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec40742b0 con 0x7f6ec41e66c0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.953+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ebc0137c0 con 0x7f6ec41e66c0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.953+0000 7f6ebaffd700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ea80682f0 0x7f6ea806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.953+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6ebc0866f0 con 0x7f6ec41e66c0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.953+0000 7f6eca092700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ea80682f0 0x7f6ea806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.953+0000 7f6eca092700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ea80682f0 0x7f6ea806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40090b0 tx=0x7f6eb4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:30.957+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ebc052c20 con 0x7f6ec41e66c0 2023-11-28T11:09:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.142+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ec406fb90 con 0x7f6ec41e66c0 2023-11-28T11:09:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.143+0000 7f6ebaffd700 1 -- 172.21.15.83:0/1241390360 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ebc056240 con 0x7f6ec41e66c0 2023-11-28T11:09:31.144 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:31.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ea80682f0 msgr2=0x7f6ea806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ea80682f0 0x7f6ea806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40090b0 tx=0x7f6eb4008030 comp rx=0 tx=0).stop 2023-11-28T11:09:31.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 msgr2=0x7f6ec4150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 0x7f6ec4150900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ec406f640 tx=0x7f6ebc00e440 comp rx=0 tx=0).stop 2023-11-28T11:09:31.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 shutdown_connections 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6ea80682f0 0x7f6ea806a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ec41e66c0 0x7f6ec4150900 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ec41e62f0 0x7f6ec41e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 --2- 172.21.15.83:0/1241390360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ec41e3df0 0x7f6ec41503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 >> 172.21.15.83:0/1241390360 conn(0x7f6eb0003a00 msgr2=0x7f6ec415ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 shutdown_connections 2023-11-28T11:09:31.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.145+0000 7f6ecc2f6700 1 -- 172.21.15.83:0/1241390360 wait complete. 2023-11-28T11:09:31.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:31.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.281+0000 7ff94f985700 1 Processor -- start 2023-11-28T11:09:31.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.281+0000 7ff94f985700 1 -- start start 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94f985700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff94815e5c0 0x7ff94815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94f985700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff94815b3b0 0x7ff9480d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94f985700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff948151a30 con 0x7ff94815b850 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94f985700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9480cb460 con 0x7ff94815e5c0 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94f985700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9480cb740 con 0x7ff94815b3b0 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff94815e5c0 0x7ff94815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff94815e5c0 0x7ff94815e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50250/0 (socket says 172.21.15.83:50250) 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 -- 172.21.15.83:0/425388849 learned_addr learned my addr 172.21.15.83:0/425388849 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:31.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94cf20700 1 --2- 172.21.15.83:0/425388849 >> v2:172.21.15.90:3300/0 conn(0x7ff94815b3b0 0x7ff9480d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 -- 172.21.15.83:0/425388849 >> v2:172.21.15.90:3300/0 conn(0x7ff94815b3b0 msgr2=0x7ff9480d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 --2- 172.21.15.83:0/425388849 >> v2:172.21.15.90:3300/0 conn(0x7ff94815b3b0 0x7ff9480d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 -- 172.21.15.83:0/425388849 >> v1:172.21.15.90:6789/0 conn(0x7ff94815b850 legacy=0x7ff9480ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 -- 172.21.15.83:0/425388849 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94815abb0 con 0x7ff94815e5c0 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff94d721700 1 --2- 172.21.15.83:0/425388849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815e5c0 0x7ff94815e990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff94806f860 tx=0x7ff944009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a45127a306cc5fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff93ffff700 1 -- 172.21.15.83:0/425388849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944004230 con 0x7ff94815e5c0 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff93ffff700 1 -- 172.21.15.83:0/425388849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944004390 con 0x7ff94815e5c0 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.282+0000 7ff93ffff700 1 -- 172.21.15.83:0/425388849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440036a0 con 0x7ff94815e5c0 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 -- 172.21.15.83:0/425388849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815e5c0 msgr2=0x7ff94815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 --2- 172.21.15.83:0/425388849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815e5c0 0x7ff94815e990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff94806f860 tx=0x7ff944009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 -- 172.21.15.83:0/425388849 shutdown_connections 2023-11-28T11:09:31.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 --2- 172.21.15.83:0/425388849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815e5c0 0x7ff94815e990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 --2- 172.21.15.83:0/425388849 >> v2:172.21.15.90:3300/0 conn(0x7ff94815b3b0 0x7ff9480d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 -- 172.21.15.83:0/425388849 >> 172.21.15.83:0/425388849 conn(0x7ff94806ea50 msgr2=0x7ff948070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 -- 172.21.15.83:0/425388849 shutdown_connections 2023-11-28T11:09:31.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.283+0000 7ff94f985700 1 -- 172.21.15.83:0/425388849 wait complete. 2023-11-28T11:09:31.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.284+0000 7ff94f985700 1 Processor -- start 2023-11-28T11:09:31.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.284+0000 7ff94f985700 1 -- start start 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94f985700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 0x7ff9480c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94f985700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff94815e5c0 0x7ff9480b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94f985700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff948146060 0x7ff9480c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94f985700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480b6820 con 0x7ff94815b850 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94f985700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480b6a30 con 0x7ff94815e5c0 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94f985700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480b3c40 con 0x7ff948146060 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 0x7ff9480c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 0x7ff9480c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50256/0 (socket says 172.21.15.83:50256) 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 -- 172.21.15.83:0/3331206743 learned_addr learned my addr 172.21.15.83:0/3331206743 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 -- 172.21.15.83:0/3331206743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff948146060 msgr2=0x7ff9480c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff948146060 0x7ff9480c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 -- 172.21.15.83:0/3331206743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff94815e5c0 msgr2=0x7ff9480b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff94815e5c0 0x7ff9480b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.285+0000 7ff94d721700 1 -- 172.21.15.83:0/3331206743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff944009550 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.286+0000 7ff94d721700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 0x7ff9480c73c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff944009130 tx=0x7ff944004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.286+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944003d40 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.286+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94401f070 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.286+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94815eed0 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.286+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944011870 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.286+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94815ae90 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.287+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9480cba20 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.287+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff944017020 con 0x7ff94815b850 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.288+0000 7ff93dffb700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff92c0682f0 0x7ff92c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.288+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff9440876c0 con 0x7ff94815b850 2023-11-28T11:09:31.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.288+0000 7ff94cf20700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff92c0682f0 0x7ff92c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.288+0000 7ff94cf20700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff92c0682f0 0x7ff92c06a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff948149c60 tx=0x7ff938008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.292+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff944053bf0 con 0x7ff94815b850 2023-11-28T11:09:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.473+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9480742b0 con 0x7ff94815b850 2023-11-28T11:09:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.474+0000 7ff93dffb700 1 -- 172.21.15.83:0/3331206743 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff944057210 con 0x7ff94815b850 2023-11-28T11:09:31.475 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff92c0682f0 msgr2=0x7ff92c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff92c0682f0 0x7ff92c06a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff948149c60 tx=0x7ff938008040 comp rx=0 tx=0).stop 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 msgr2=0x7ff9480c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 0x7ff9480c73c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff944009130 tx=0x7ff944004750 comp rx=0 tx=0).stop 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 shutdown_connections 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff948146060 0x7ff9480c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff92c0682f0 0x7ff92c06a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94815b850 0x7ff9480c73c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 --2- 172.21.15.83:0/3331206743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff94815e5c0 0x7ff9480b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 >> 172.21.15.83:0/3331206743 conn(0x7ff94815b3b0 msgr2=0x7ff9480cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 shutdown_connections 2023-11-28T11:09:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.476+0000 7ff94f985700 1 -- 172.21.15.83:0/3331206743 wait complete. 2023-11-28T11:09:31.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:31.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.615+0000 7f1ea5fca700 1 Processor -- start 2023-11-28T11:09:31.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.615+0000 7f1ea5fca700 1 -- start start 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1ea5fca700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ea00cd880 0x7f1ea00d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1ea5fca700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ea00b0240 0x7f1ea006bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1ea5fca700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea006c410 0x7f1ea01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1ea5fca700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea006f9a0 con 0x7f1ea00cd880 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1ea5fca700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea006fc80 con 0x7f1ea00b0240 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1ea5fca700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea006ff60 con 0x7f1ea006c410 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ea00cd880 0x7f1ea00d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ea00cd880 0x7f1ea00d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50264/0 (socket says 172.21.15.83:50264) 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/2210717006 learned_addr learned my addr 172.21.15.83:0/2210717006 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9ffff700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea006c410 0x7f1ea01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9effd700 1 --2- 172.21.15.83:0/2210717006 >> v2:172.21.15.90:3300/0 conn(0x7f1ea00b0240 0x7f1ea006bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/2210717006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea006c410 msgr2=0x7f1ea01608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea006c410 0x7f1ea01608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/2210717006 >> v2:172.21.15.90:3300/0 conn(0x7f1ea00b0240 msgr2=0x7f1ea006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 --2- 172.21.15.83:0/2210717006 >> v2:172.21.15.90:3300/0 conn(0x7f1ea00b0240 0x7f1ea006bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/2210717006 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea01603b0 con 0x7f1ea00cd880 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9ffff700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea006c410 0x7f1ea01608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.616+0000 7f1e9f7fe700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea00cd880 0x7f1ea00d0710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00c4860 tx=0x7f1e88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be61b2208666ef5e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1e9e7fc700 1 -- 172.21.15.83:0/2210717006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8800b870 con 0x7f1ea00cd880 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1e9e7fc700 1 -- 172.21.15.83:0/2210717006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8800b9d0 con 0x7f1ea00cd880 2023-11-28T11:09:31.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1e9e7fc700 1 -- 172.21.15.83:0/2210717006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8800bd60 con 0x7f1ea00cd880 2023-11-28T11:09:31.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 -- 172.21.15.83:0/2210717006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea00cd880 msgr2=0x7f1ea00d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea00cd880 0x7f1ea00d0710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00c4860 tx=0x7f1e88009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 -- 172.21.15.83:0/2210717006 shutdown_connections 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea006c410 0x7f1ea01608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/2210717006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea00cd880 0x7f1ea00d0710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/2210717006 >> v2:172.21.15.90:3300/0 conn(0x7f1ea00b0240 0x7f1ea006bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 -- 172.21.15.83:0/2210717006 >> 172.21.15.83:0/2210717006 conn(0x7f1ea006de20 msgr2=0x7f1ea006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 -- 172.21.15.83:0/2210717006 shutdown_connections 2023-11-28T11:09:31.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.617+0000 7f1ea5fca700 1 -- 172.21.15.83:0/2210717006 wait complete. 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.619+0000 7f1ea5fca700 1 Processor -- start 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.619+0000 7f1ea5fca700 1 -- start start 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1ea5fca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 0x7f1ea01577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1ea5fca700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ea00b0240 0x7f1ea01f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1ea5fca700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea00cd880 0x7f1ea00c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1ea5fca700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01589b0 con 0x7f1ea006c410 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1ea5fca700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea0157da0 con 0x7f1ea00b0240 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1ea5fca700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea0158080 con 0x7f1ea00cd880 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 0x7f1ea01577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 0x7f1ea01577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50270/0 (socket says 172.21.15.83:50270) 2023-11-28T11:09:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/271189218 learned_addr learned my addr 172.21.15.83:0/271189218 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9effd700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ea00b0240 0x7f1ea01f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9ffff700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea00cd880 0x7f1ea00c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea00cd880 msgr2=0x7f1ea00c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea00cd880 0x7f1ea00c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ea00b0240 msgr2=0x7f1ea01f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ea00b0240 0x7f1ea01f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 -- 172.21.15.83:0/271189218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e88009550 con 0x7f1ea006c410 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9ffff700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea00cd880 0x7f1ea00c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e9f7fe700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 0x7f1ea01577c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000c00 tx=0x7f1e88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.620+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88004540 con 0x7f1ea006c410 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.621+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e880036a0 con 0x7f1ea006c410 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.621+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea00cdc50 con 0x7f1ea006c410 2023-11-28T11:09:31.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.621+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88003a50 con 0x7f1ea006c410 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.621+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea006d670 con 0x7f1ea006c410 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.621+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea00cdde0 con 0x7f1ea006c410 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.622+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1e8800f050 con 0x7f1ea006c410 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.622+0000 7f1e97fff700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1e800682f0 0x7f1e8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.622+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1e88087810 con 0x7f1ea006c410 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.622+0000 7f1e9effd700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1e800682f0 0x7f1e8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.623+0000 7f1e9effd700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1e800682f0 0x7f1e8006a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00c77a0 tx=0x7f1e90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.627+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1e88053d40 con 0x7f1ea006c410 2023-11-28T11:09:31.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.807+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ea00742b0 con 0x7f1ea006c410 2023-11-28T11:09:31.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.807+0000 7f1e97fff700 1 -- 172.21.15.83:0/271189218 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e88057360 con 0x7f1ea006c410 2023-11-28T11:09:31.809 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:31.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1e800682f0 msgr2=0x7f1e8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1e800682f0 0x7f1e8006a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00c77a0 tx=0x7f1e90008040 comp rx=0 tx=0).stop 2023-11-28T11:09:31.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 msgr2=0x7f1ea01577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 0x7f1ea01577c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000c00 tx=0x7f1e88004750 comp rx=0 tx=0).stop 2023-11-28T11:09:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 shutdown_connections 2023-11-28T11:09:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ea00cd880 0x7f1ea00c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1e800682f0 0x7f1e8006a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ea006c410 0x7f1ea01577c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 --2- 172.21.15.83:0/271189218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ea00b0240 0x7f1ea01f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 >> 172.21.15.83:0/271189218 conn(0x7f1ea006de20 msgr2=0x7f1ea015a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 shutdown_connections 2023-11-28T11:09:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.809+0000 7f1ea5fca700 1 -- 172.21.15.83:0/271189218 wait complete. 2023-11-28T11:09:31.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.945+0000 7f6205e2d700 1 Processor -- start 2023-11-28T11:09:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.946+0000 7f6205e2d700 1 -- start start 2023-11-28T11:09:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.946+0000 7f6205e2d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f62000aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.946+0000 7f6205e2d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f620006e370 0x7f620006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.946+0000 7f6205e2d700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620006d4a0 con 0x7f62000b4bb0 2023-11-28T11:09:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.946+0000 7f6205e2d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62000b5020 con 0x7f620006e370 2023-11-28T11:09:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.946+0000 7f6205e2d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000b5ae0 con 0x7f620015c890 2023-11-28T11:09:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f620006e370 0x7f620006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f620006e370 0x7f620006b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50280/0 (socket says 172.21.15.83:50280) 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 -- 172.21.15.83:0/1413738290 learned_addr learned my addr 172.21.15.83:0/1413738290 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61ff7fe700 1 --2- 172.21.15.83:0/1413738290 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f62000aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 -- 172.21.15.83:0/1413738290 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 msgr2=0x7f62000aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 --2- 172.21.15.83:0/1413738290 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f62000aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 -- 172.21.15.83:0/1413738290 >> v1:172.21.15.90:6789/0 conn(0x7f62000b4bb0 legacy=0x7f620006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 -- 172.21.15.83:0/1413738290 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620006c400 con 0x7f620006e370 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fffff700 1 --2- 172.21.15.83:0/1413738290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f620006b150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009e50 tx=0x7f61f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce5281b5dc46c36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fe7fc700 1 -- 172.21.15.83:0/1413738290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f800e070 con 0x7f620006e370 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fe7fc700 1 -- 172.21.15.83:0/1413738290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f800b9c0 con 0x7f620006e370 2023-11-28T11:09:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.947+0000 7f61fe7fc700 1 -- 172.21.15.83:0/1413738290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f800bd70 con 0x7f620006e370 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 -- 172.21.15.83:0/1413738290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 msgr2=0x7f620006b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1413738290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f620006b150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009e50 tx=0x7f61f800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 -- 172.21.15.83:0/1413738290 shutdown_connections 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1413738290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f620006b150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1413738290 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f62000aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 -- 172.21.15.83:0/1413738290 >> 172.21.15.83:0/1413738290 conn(0x7f62000b2b60 msgr2=0x7f62000b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 -- 172.21.15.83:0/1413738290 shutdown_connections 2023-11-28T11:09:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.948+0000 7f6205e2d700 1 -- 172.21.15.83:0/1413738290 wait complete. 2023-11-28T11:09:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.949+0000 7f6205e2d700 1 Processor -- start 2023-11-28T11:09:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.949+0000 7f6205e2d700 1 -- start start 2023-11-28T11:09:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f6205e2d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f62001f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f6205e2d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62000b4bb0 0x7f620009c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f6205e2d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f620009c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f6205e2d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200147b90 con 0x7f620006e370 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f6205e2d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000c5e10 con 0x7f62000b4bb0 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f6205e2d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000c60f0 con 0x7f620015c890 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f62001f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f62001f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50284/0 (socket says 172.21.15.83:50284) 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 -- 172.21.15.83:0/1151976396 learned_addr learned my addr 172.21.15.83:0/1151976396 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61fffff700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f620009c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61feffd700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62000b4bb0 0x7f620009c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 -- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 msgr2=0x7f620009c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f620009c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 -- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62000b4bb0 msgr2=0x7f620009c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62000b4bb0 0x7f620009c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 -- 172.21.15.83:0/1151976396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f8009b30 con 0x7f620006e370 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.950+0000 7f61ff7fe700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f62001f1a70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f61e800d7d0 tx=0x7f61e800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.951+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8017070 con 0x7f620006e370 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.951+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8005a20 con 0x7f620006e370 2023-11-28T11:09:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.951+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62000b5520 con 0x7f620006e370 2023-11-28T11:09:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.951+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8005bd0 con 0x7f620006e370 2023-11-28T11:09:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.951+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62000b5240 con 0x7f620006e370 2023-11-28T11:09:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.952+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62000b5800 con 0x7f620006e370 2023-11-28T11:09:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.952+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f61e8005d30 con 0x7f620006e370 2023-11-28T11:09:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.953+0000 7f61f7fff700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61e00682f0 0x7f61e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.953+0000 7f61feffd700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61e00682f0 0x7f61e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.953+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f61e8085ac0 con 0x7f620006e370 2023-11-28T11:09:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.953+0000 7f61feffd700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61e00682f0 0x7f61e006a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f620009d000 tx=0x7f61f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:31.957+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61e8051ff0 con 0x7f620006e370 2023-11-28T11:09:32.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.140+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6200148430 con 0x7f620006e370 2023-11-28T11:09:32.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.140+0000 7f61f7fff700 1 -- 172.21.15.83:0/1151976396 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f61e8055610 con 0x7f620006e370 2023-11-28T11:09:32.142 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61e00682f0 msgr2=0x7f61e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61e00682f0 0x7f61e006a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f620009d000 tx=0x7f61f0009040 comp rx=0 tx=0).stop 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 msgr2=0x7f62001f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f62001f1a70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f61e800d7d0 tx=0x7f61e800dae0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 shutdown_connections 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f620015c890 0x7f620009c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61e00682f0 0x7f61e006a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620006e370 0x7f62001f1a70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 --2- 172.21.15.83:0/1151976396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62000b4bb0 0x7f620009c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 >> 172.21.15.83:0/1151976396 conn(0x7f62000b2b60 msgr2=0x7f62000d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 shutdown_connections 2023-11-28T11:09:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.142+0000 7f6205e2d700 1 -- 172.21.15.83:0/1151976396 wait complete. 2023-11-28T11:09:32.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.273+0000 7fcbac77f700 1 Processor -- start 2023-11-28T11:09:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.273+0000 7fcbac77f700 1 -- start start 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcbac77f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcba40cf0d0 0x7fcba406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcbac77f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba40cf540 con 0x7fcba40b18a0 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcbac77f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba40cf790 con 0x7fcba4159830 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcbac77f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba415d480 con 0x7fcba40cf0d0 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcbaad1c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcba40b18a0 0x7fcba40cd9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48154/0 (socket says 172.21.15.83:48154) 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcbaad1c700 1 -- 172.21.15.83:0/1922637812 learned_addr learned my addr 172.21.15.83:0/1922637812 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcba9d1a700 1 --2- 172.21.15.83:0/1922637812 >> v2:172.21.15.90:3300/0 conn(0x7fcba40cf0d0 0x7fcba406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654470852 0 0) 0x7fcba40cf540 con 0x7fcba40b18a0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.274+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb900035b0 con 0x7fcba40b18a0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041640090 0 0) 0x7fcba40cf790 con 0x7fcba4159830 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba40cf540 con 0x7fcba4159830 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9d1a700 1 -- 172.21.15.83:0/1922637812 >> v1:172.21.15.90:6789/0 conn(0x7fcba4159830 legacy=0x7fcba415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3437852734 0 0) 0x7fcb900035b0 con 0x7fcba40b18a0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9d1a700 1 -- 172.21.15.83:0/1922637812 >> v1:172.21.15.83:6789/0 conn(0x7fcba40b18a0 legacy=0x7fcba40cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9d1a700 1 -- 172.21.15.83:0/1922637812 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40cfa70 con 0x7fcba40cf0d0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9d1a700 1 --2- 172.21.15.83:0/1922637812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba406c5f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009840 tx=0x7fcba0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46527cd209b8d96f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0019f0 con 0x7fcba40cf0d0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0004030 con 0x7fcba40cf0d0 2023-11-28T11:09:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcba9519700 1 -- 172.21.15.83:0/1922637812 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00043c0 con 0x7fcba40cf0d0 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcbac77f700 1 -- 172.21.15.83:0/1922637812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 msgr2=0x7fcba406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.275+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1922637812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba406c5f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009840 tx=0x7fcba0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.276+0000 7fcbac77f700 1 -- 172.21.15.83:0/1922637812 shutdown_connections 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.276+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1922637812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba406c5f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.276+0000 7fcbac77f700 1 -- 172.21.15.83:0/1922637812 >> 172.21.15.83:0/1922637812 conn(0x7fcba406e9a0 msgr2=0x7fcba40cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.276+0000 7fcbac77f700 1 -- 172.21.15.83:0/1922637812 shutdown_connections 2023-11-28T11:09:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.276+0000 7fcbac77f700 1 -- 172.21.15.83:0/1922637812 wait complete. 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.277+0000 7fcbac77f700 1 Processor -- start 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.277+0000 7fcbac77f700 1 -- start start 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.277+0000 7fcbac77f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 0x7fcba40d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.277+0000 7fcbac77f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba4154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbac77f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcba4159830 0x7fcba4155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbac77f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415b2b0 con 0x7fcba40b18a0 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbac77f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415acf0 con 0x7fcba40cf0d0 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbac77f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415afd0 con 0x7fcba4159830 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 0x7fcba40d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 0x7fcba40d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50296/0 (socket says 172.21.15.83:50296) 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 -- 172.21.15.83:0/1909978577 learned_addr learned my addr 172.21.15.83:0/1909978577 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcba9d1a700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba4154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaad1c700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcba4159830 0x7fcba4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 -- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcba4159830 msgr2=0x7fcba4155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcba4159830 0x7fcba4155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 -- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 msgr2=0x7fcba4154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba4154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 -- 172.21.15.83:0/1909978577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00094a0 con 0x7fcba40b18a0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcba9d1a700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba4154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbaa51b700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 0x7fcba40d0430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c520 tx=0x7fcb9400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940052d0 con 0x7fcba40b18a0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94005430 con 0x7fcba40b18a0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba415d970 con 0x7fcba40b18a0 2023-11-28T11:09:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940038c0 con 0x7fcba40b18a0 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.278+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba415d690 con 0x7fcba40b18a0 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.279+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba4159c00 con 0x7fcba40b18a0 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.279+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb94019030 con 0x7fcba40b18a0 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.280+0000 7fcb9affd700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcb880682f0 0x7fcb8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.280+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcb94085490 con 0x7fcba40b18a0 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.280+0000 7fcba9d1a700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcb880682f0 0x7fcb8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.280+0000 7fcba9d1a700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcb880682f0 0x7fcb8806a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcba40c6130 tx=0x7fcba0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.284+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb94051940 con 0x7fcba40b18a0 2023-11-28T11:09:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.507+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcba4074320 con 0x7fcba40b18a0 2023-11-28T11:09:32.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.507+0000 7fcb9affd700 1 -- 172.21.15.83:0/1909978577 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fcb94054f60 con 0x7fcba40b18a0 2023-11-28T11:09:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:32.509 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:51.826077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcb880682f0 msgr2=0x7fcb8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcb880682f0 0x7fcb8806a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcba40c6130 tx=0x7fcba0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 msgr2=0x7fcba40d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 0x7fcba40d0430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c520 tx=0x7fcb9400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 shutdown_connections 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcba4159830 0x7fcba4155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcb880682f0 0x7fcb8806a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba40b18a0 0x7fcba40d0430 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 --2- 172.21.15.83:0/1909978577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcba40cf0d0 0x7fcba4154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 >> 172.21.15.83:0/1909978577 conn(0x7fcba406e9a0 msgr2=0x7fcba40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 shutdown_connections 2023-11-28T11:09:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.509+0000 7fcbac77f700 1 -- 172.21.15.83:0/1909978577 wait complete. 2023-11-28T11:09:32.513 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2023-11-28T11:09:32.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:32.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.641+0000 7f7dbf35d700 1 Processor -- start 2023-11-28T11:09:32.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.641+0000 7f7dbf35d700 1 -- start start 2023-11-28T11:09:32.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbf35d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7db815d0e0 0x7f7db80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbf35d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbf35d700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db80cbfb0 con 0x7f7db815cd10 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbf35d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db806cee0 con 0x7f7db815d0e0 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbf35d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80cb710 con 0x7f7db80b1fb0 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbc8f8700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7db815d0e0 0x7f7db80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbd0f9700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f7db815cd10 0x7f7db80b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:46510/0 (socket says 172.21.15.83:46510) 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbd8fa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.642+0000 7f7dbd0f9700 1 -- 172.21.15.83:0/2687487773 learned_addr learned my addr 172.21.15.83:0/2687487773 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 msgr2=0x7f7db80cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7dbc8f8700 1 --2- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/2687487773 >> v1:172.21.15.90:6789/0 conn(0x7f7db815cd10 legacy=0x7f7db80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/2687487773 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80cb4e0 con 0x7f7db815d0e0 2023-11-28T11:09:32.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7dbc8f8700 1 --2- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db80ca970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009840 tx=0x7f7db4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3dc2b2c27e600a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7daffff700 1 -- 172.21.15.83:0/2687487773 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4011830 con 0x7f7db815d0e0 2023-11-28T11:09:32.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7daffff700 1 -- 172.21.15.83:0/2687487773 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4011990 con 0x7f7db815d0e0 2023-11-28T11:09:32.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.643+0000 7f7daffff700 1 -- 172.21.15.83:0/2687487773 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4011d20 con 0x7f7db815d0e0 2023-11-28T11:09:32.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 -- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 msgr2=0x7f7db80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db80ca970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009840 tx=0x7f7db4009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:32.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 -- 172.21.15.83:0/2687487773 shutdown_connections 2023-11-28T11:09:32.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/2687487773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db80ca970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 -- 172.21.15.83:0/2687487773 >> 172.21.15.83:0/2687487773 conn(0x7f7db806e9a0 msgr2=0x7f7db80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 -- 172.21.15.83:0/2687487773 shutdown_connections 2023-11-28T11:09:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.644+0000 7f7dbf35d700 1 -- 172.21.15.83:0/2687487773 wait complete. 2023-11-28T11:09:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.645+0000 7f7dbf35d700 1 Processor -- start 2023-11-28T11:09:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.645+0000 7f7dbf35d700 1 -- start start 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbf35d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbf35d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 0x7f7db80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbf35d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db8159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbf35d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db8151cb0 con 0x7f7db815cd10 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbf35d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80abcc0 con 0x7f7db815d0e0 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbf35d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db815af40 con 0x7f7db80b1fb0 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 0x7f7db80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 0x7f7db80b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50312/0 (socket says 172.21.15.83:50312) 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/646862407 learned_addr learned my addr 172.21.15.83:0/646862407 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbd8fa700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db8159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbd0f9700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 msgr2=0x7f7db80b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 msgr2=0x7f7db8159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db8159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 -- 172.21.15.83:0/646862407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40094a0 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbd8fa700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dbc8f8700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 0x7f7db80b3190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7db40090b0 tx=0x7f7db4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.646+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db40036a0 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.647+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db401e070 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.647+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db80cbcd0 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.647+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4003a40 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.647+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db80cb9f0 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.648+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db806b9b0 con 0x7f7db815cd10 2023-11-28T11:09:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.648+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7db400f050 con 0x7f7db815cd10 2023-11-28T11:09:32.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.648+0000 7f7dadffb700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7d9c0682f0 0x7f7d9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.649+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7db4086e30 con 0x7f7db815cd10 2023-11-28T11:09:32.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.649+0000 7f7dbd0f9700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7d9c0682f0 0x7f7d9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.649+0000 7f7dbd0f9700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7d9c0682f0 0x7f7d9c06a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005910 tx=0x7f7da8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.653+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7db4053360 con 0x7f7db815cd10 2023-11-28T11:09:32.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.837+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7db80b27b0 con 0x7f7db815cd10 2023-11-28T11:09:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.839+0000 7f7dadffb700 1 -- 172.21.15.83:0/646862407 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f7db4015090 con 0x7f7db815cd10 2023-11-28T11:09:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:32.840 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:25.004035+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7d9c0682f0 msgr2=0x7f7d9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7d9c0682f0 0x7f7d9c06a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005910 tx=0x7f7da8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 msgr2=0x7f7db80b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 0x7f7db80b3190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7db40090b0 tx=0x7f7db4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 shutdown_connections 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7db815d0e0 0x7f7db8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7db80b1fb0 0x7f7db80b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7d9c0682f0 0x7f7d9c06a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 --2- 172.21.15.83:0/646862407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db815cd10 0x7f7db80b3190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 >> 172.21.15.83:0/646862407 conn(0x7f7db806e9a0 msgr2=0x7f7db80cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 shutdown_connections 2023-11-28T11:09:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.841+0000 7f7dbf35d700 1 -- 172.21.15.83:0/646862407 wait complete. 2023-11-28T11:09:32.855 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:09:32.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:32.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.974+0000 7fdcfece5700 1 Processor -- start 2023-11-28T11:09:32.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.974+0000 7fdcfece5700 1 -- start start 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfece5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdcf815cd10 0x7fdcf80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfece5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf80cbfb0 con 0x7fdcf815d0e0 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfece5700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf80cb710 con 0x7fdcf80b1fb0 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfece5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf80cb9f0 con 0x7fdcf815cd10 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfca81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdcf815cd10 0x7fdcf80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfca81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdcf815cd10 0x7fdcf80ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50318/0 (socket says 172.21.15.83:50318) 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcf7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdcf815d0e0 0x7fdcf80b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48178/0 (socket says 172.21.15.83:48178) 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfca81700 1 -- 172.21.15.83:0/2069257635 learned_addr learned my addr 172.21.15.83:0/2069257635 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfca81700 1 -- 172.21.15.83:0/2069257635 >> v1:172.21.15.90:6789/0 conn(0x7fdcf80b1fb0 legacy=0x7fdcf806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfca81700 1 -- 172.21.15.83:0/2069257635 >> v1:172.21.15.83:6789/0 conn(0x7fdcf815d0e0 legacy=0x7fdcf80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcfca81700 1 -- 172.21.15.83:0/2069257635 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80cbcd0 con 0x7fdcf815cd10 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.975+0000 7fdcf77fe700 1 -- 172.21.15.83:0/2069257635 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252815245 0 0) 0x7fdcf80cbfb0 con 0x7fdcf815d0e0 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfca81700 1 --2- 172.21.15.83:0/2069257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ca970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0098f0 tx=0x7fdcec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6e5ec01e6755b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcf77fe700 1 -- 172.21.15.83:0/2069257635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004230 con 0x7fdcf815cd10 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcf77fe700 1 -- 172.21.15.83:0/2069257635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec004390 con 0x7fdcf815cd10 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcf77fe700 1 -- 172.21.15.83:0/2069257635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0036a0 con 0x7fdcf815cd10 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 -- 172.21.15.83:0/2069257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 msgr2=0x7fdcf80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 --2- 172.21.15.83:0/2069257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ca970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0098f0 tx=0x7fdcec009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 -- 172.21.15.83:0/2069257635 shutdown_connections 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 --2- 172.21.15.83:0/2069257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ca970 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 -- 172.21.15.83:0/2069257635 >> 172.21.15.83:0/2069257635 conn(0x7fdcf806e9a0 msgr2=0x7fdcf80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 -- 172.21.15.83:0/2069257635 shutdown_connections 2023-11-28T11:09:32.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.976+0000 7fdcfece5700 1 -- 172.21.15.83:0/2069257635 wait complete. 2023-11-28T11:09:32.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.977+0000 7fdcfece5700 1 Processor -- start 2023-11-28T11:09:32.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 -- start start 2023-11-28T11:09:32.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf80b1fb0 0x7fdcf81e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf815d0e0 0x7fdcf8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80ae9c0 con 0x7fdcf815cd10 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8159650 con 0x7fdcf80b1fb0 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.978+0000 7fdcfece5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8159930 con 0x7fdcf815d0e0 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50326/0 (socket says 172.21.15.83:50326) 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 -- 172.21.15.83:0/3194375103 learned_addr learned my addr 172.21.15.83:0/3194375103 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcfca81700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf80b1fb0 0x7fdcf81e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcfd282700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf815d0e0 0x7fdcf8158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 -- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf815d0e0 msgr2=0x7fdcf8158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf815d0e0 0x7fdcf8158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 -- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf80b1fb0 msgr2=0x7fdcf81e69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf80b1fb0 0x7fdcf81e69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 -- 172.21.15.83:0/3194375103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec009550 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcfd282700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf815d0e0 0x7fdcf8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcfca81700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf80b1fb0 0x7fdcf81e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf7fff700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ae480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c550 tx=0x7fdce400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4004c80 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4004e00 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4007310 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf806b660 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.979+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf806b9b0 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.980+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf80cd9f0 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.980+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdce401a020 con 0x7fdcf815cd10 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.981+0000 7fdcf57fa700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdcdc0682f0 0x7fdcdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.981+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdce4085b70 con 0x7fdcf815cd10 2023-11-28T11:09:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.981+0000 7fdcfca81700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdcdc0682f0 0x7fdcdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.981+0000 7fdcfca81700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdcdc0682f0 0x7fdcdc06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009130 tx=0x7fdcec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:32.985+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdce4052150 con 0x7fdcf815cd10 2023-11-28T11:09:33.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.214+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcf8159790 con 0x7fdcf815cd10 2023-11-28T11:09:33.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.214+0000 7fdcf57fa700 1 -- 172.21.15.83:0/3194375103 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fdcf8159790 con 0x7fdcf815cd10 2023-11-28T11:09:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:33.216 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:51.826077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:33.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdcdc0682f0 msgr2=0x7fdcdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdcdc0682f0 0x7fdcdc06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009130 tx=0x7fdcec008030 comp rx=0 tx=0).stop 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 msgr2=0x7fdcf80ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ae480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c550 tx=0x7fdce400c910 comp rx=0 tx=0).stop 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 shutdown_connections 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf815d0e0 0x7fdcf8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf80b1fb0 0x7fdcf81e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdcdc0682f0 0x7fdcdc06a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 --2- 172.21.15.83:0/3194375103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf815cd10 0x7fdcf80ae480 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 >> 172.21.15.83:0/3194375103 conn(0x7fdcf806e9a0 msgr2=0x7fdcf80caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 shutdown_connections 2023-11-28T11:09:33.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.216+0000 7fdcfece5700 1 -- 172.21.15.83:0/3194375103 wait complete. 2023-11-28T11:09:33.219 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2023-11-28T11:09:33.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.348+0000 7f965cc7f700 1 Processor -- start 2023-11-28T11:09:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.348+0000 7f965cc7f700 1 -- start start 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965cc7f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 0x7f965806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965cc7f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f965806b8b0 0x7f965806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965cc7f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965806b490 con 0x7f965815aad0 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965cc7f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96580cb220 con 0x7f965806b8b0 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965cc7f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965815aea0 con 0x7f965806ab00 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f965815aad0 0x7f96580b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48192/0 (socket says 172.21.15.83:48192) 2023-11-28T11:09:33.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965659c700 1 -- 172.21.15.83:0/3100274426 learned_addr learned my addr 172.21.15.83:0/3100274426 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f9656d9d700 1 --2- 172.21.15.83:0/3100274426 >> v2:172.21.15.90:3300/0 conn(0x7f965806b8b0 0x7f965806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f9655d9b700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 0x7f965806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141476054 0 0) 0x7f9648001110 con 0x7f965815aad0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96400035b0 con 0x7f965815aad0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3732662257 0 0) 0x7f96400035b0 con 0x7f965815aad0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 msgr2=0x7f965806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 0x7f965806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 >> v2:172.21.15.90:3300/0 conn(0x7f965806b8b0 msgr2=0x7f965806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 --2- 172.21.15.83:0/3100274426 >> v2:172.21.15.90:3300/0 conn(0x7f965806b8b0 0x7f965806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580ca610 con 0x7f965815aad0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f9648001800 con 0x7f965815aad0 2023-11-28T11:09:33.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f9656d9d700 1 --2- 172.21.15.83:0/3100274426 >> v2:172.21.15.90:3300/0 conn(0x7f965806b8b0 0x7f965806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 >> v1:172.21.15.83:6789/0 conn(0x7f965815aad0 legacy=0x7f96580b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9640003a00 0x7f9640003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 0x7f9640004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9640004d90 0x7f9640005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cb220 con 0x7f9640003a00 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965815aea0 con 0x7f9640004d90 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.349+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964000ba10 con 0x7f965806ab00 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9640003a00 0x7f9640003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f9655d9b700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 0x7f9640004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f9656d9d700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9640004d90 0x7f9640005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 -- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 msgr2=0x7f9640004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f965806ab00 0x7f9640004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 -- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9640004d90 msgr2=0x7f9640005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9640004d90 0x7f9640005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 -- 172.21.15.83:0/3100274426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400035b0 con 0x7f9640003a00 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965659c700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9640003a00 0x7f9640003e10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f964800c2f0 tx=0x7f9640005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90d9af4543d67b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800ce30 con 0x7f9640003a00 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9648005960 con 0x7f9640003a00 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.350+0000 7f965559a700 1 -- 172.21.15.83:0/3100274426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648005d10 con 0x7f9640003a00 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f965cc7f700 1 -- 172.21.15.83:0/3100274426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9640003a00 msgr2=0x7f9640003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f965cc7f700 1 --2- 172.21.15.83:0/3100274426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9640003a00 0x7f9640003e10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f964800c2f0 tx=0x7f9640005880 comp rx=0 tx=0).stop 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f9656d9d700 1 -- 172.21.15.83:0/3100274426 reap_dead start 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f965cc7f700 1 -- 172.21.15.83:0/3100274426 shutdown_connections 2023-11-28T11:09:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f965cc7f700 1 -- 172.21.15.83:0/3100274426 >> 172.21.15.83:0/3100274426 conn(0x7f965806e9a0 msgr2=0x7f96580cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f965cc7f700 1 -- 172.21.15.83:0/3100274426 shutdown_connections 2023-11-28T11:09:33.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.351+0000 7f965cc7f700 1 -- 172.21.15.83:0/3100274426 wait complete. 2023-11-28T11:09:33.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.352+0000 7f965cc7f700 1 Processor -- start 2023-11-28T11:09:33.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 -- start start 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9640003a00 0x7f96580c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 0x7f965815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9658153f60 0x7f96581543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c7e00 con 0x7f9658157660 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c7560 con 0x7f9658153f60 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.353+0000 7f965cc7f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c7840 con 0x7f9640003a00 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 0x7f965815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 0x7f965815be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50346/0 (socket says 172.21.15.83:50346) 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 -- 172.21.15.83:0/2709081651 learned_addr learned my addr 172.21.15.83:0/2709081651 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f965659c700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9640003a00 0x7f96580c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9656d9d700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9658153f60 0x7f96581543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 -- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9640003a00 msgr2=0x7f96580c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9640003a00 0x7f96580c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 -- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9658153f60 msgr2=0x7f96581543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9658153f60 0x7f96581543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 -- 172.21.15.83:0/2709081651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580cabd0 con 0x7f9658157660 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9656d9d700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9658153f60 0x7f96581543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f965659c700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9640003a00 0x7f96580c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9655d9b700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 0x7f965815be90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f964c0119a0 tx=0x7f964c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00f940 con 0x7f9658157660 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00faa0 con 0x7f9658157660 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96580ca8f0 con 0x7f9658157660 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00fe30 con 0x7f9658157660 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.354+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96580caeb0 con 0x7f9658157660 2023-11-28T11:09:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.355+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9658074320 con 0x7f9658157660 2023-11-28T11:09:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.355+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f964c014440 con 0x7f9658157660 2023-11-28T11:09:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.356+0000 7f9646ffd700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96380682f0 0x7f963806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.356+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f964c086830 con 0x7f9658157660 2023-11-28T11:09:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.356+0000 7f965659c700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96380682f0 0x7f963806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.356+0000 7f965659c700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96380682f0 0x7f963806a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9648008fd0 tx=0x7f964800a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.360+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f964c052d00 con 0x7f9658157660 2023-11-28T11:09:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.586+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9658157ad0 con 0x7f9658157660 2023-11-28T11:09:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.587+0000 7f9646ffd700 1 -- 172.21.15.83:0/2709081651 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f964c056320 con 0x7f9658157660 2023-11-28T11:09:33.590 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:33.590 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2785225849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2785225849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2785225849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2325696730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2325696730},{"type":"v1","addr":"172.21.15.90:6833","nonce":2325696730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/235754448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":235754448},{"type":"v1","addr":"172.21.15.90:6835","nonce":235754448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:35.518363+0000","modified":"2023-11-28T11:08:51.826077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3706545143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3706545143},{"type":"v1","addr":"172.21.15.83:6836","nonce":3706545143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96380682f0 msgr2=0x7f963806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96380682f0 0x7f963806a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9648008fd0 tx=0x7f964800a000 comp rx=0 tx=0).stop 2023-11-28T11:09:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 msgr2=0x7f965815be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 0x7f965815be90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f964c0119a0 tx=0x7f964c011d60 comp rx=0 tx=0).stop 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 shutdown_connections 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9658153f60 0x7f96581543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9640003a00 0x7f96580c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f96380682f0 0x7f963806a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 --2- 172.21.15.83:0/2709081651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9658157660 0x7f965815be90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.588+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 >> 172.21.15.83:0/2709081651 conn(0x7f965806e9a0 msgr2=0x7f96580b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.589+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 shutdown_connections 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.589+0000 7f965cc7f700 1 -- 172.21.15.83:0/2709081651 wait complete. 2023-11-28T11:09:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2023-11-28T11:09:33.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:33.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.720+0000 7fdf1baa8700 1 Processor -- start 2023-11-28T11:09:33.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.720+0000 7fdf1baa8700 1 -- start start 2023-11-28T11:09:33.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1baa8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf1415c9c0 0x7fdf1406d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1baa8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf140b24a0 0x7fdf1415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1baa8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1406bd20 con 0x7fdf1406c1b0 2023-11-28T11:09:33.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1baa8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1406b450 con 0x7fdf1415c9c0 2023-11-28T11:09:33.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1baa8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141612e0 con 0x7fdf140b24a0 2023-11-28T11:09:33.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf19844700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf1415c9c0 0x7fdf1406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1a045700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdf1406c1b0 0x7fdf140cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48216/0 (socket says 172.21.15.83:48216) 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf19844700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf1415c9c0 0x7fdf1406d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50352/0 (socket says 172.21.15.83:50352) 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf1a045700 1 -- 172.21.15.83:0/3071011860 learned_addr learned my addr 172.21.15.83:0/3071011860 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.721+0000 7fdf19043700 1 --2- 172.21.15.83:0/3071011860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf140b24a0 0x7fdf1415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf19844700 1 -- 172.21.15.83:0/3071011860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf140b24a0 msgr2=0x7fdf1415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf19844700 1 --2- 172.21.15.83:0/3071011860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf140b24a0 0x7fdf1415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf19844700 1 -- 172.21.15.83:0/3071011860 >> v1:172.21.15.83:6789/0 conn(0x7fdf1406c1b0 legacy=0x7fdf140cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf19844700 1 -- 172.21.15.83:0/3071011860 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14161580 con 0x7fdf1415c9c0 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf18842700 1 -- 172.21.15.83:0/3071011860 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573311586 0 0) 0x7fdf1406bd20 con 0x7fdf1406c1b0 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf19844700 1 --2- 172.21.15.83:0/3071011860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf1415c9c0 0x7fdf1406d210 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009840 tx=0x7fdf04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf12c8fc788c9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf18842700 1 -- 172.21.15.83:0/3071011860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04011830 con 0x7fdf1415c9c0 2023-11-28T11:09:33.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf18842700 1 -- 172.21.15.83:0/3071011860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf04011990 con 0x7fdf1415c9c0 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf18842700 1 -- 172.21.15.83:0/3071011860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04011d20 con 0x7fdf1415c9c0 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf1baa8700 1 -- 172.21.15.83:0/3071011860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf1415c9c0 msgr2=0x7fdf1406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.722+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/3071011860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf1415c9c0 0x7fdf1406d210 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009840 tx=0x7fdf04009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.723+0000 7fdf1baa8700 1 -- 172.21.15.83:0/3071011860 shutdown_connections 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.723+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/3071011860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf1415c9c0 0x7fdf1406d210 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.723+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/3071011860 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf140b24a0 0x7fdf1415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.723+0000 7fdf1baa8700 1 -- 172.21.15.83:0/3071011860 >> 172.21.15.83:0/3071011860 conn(0x7fdf1406e9f0 msgr2=0x7fdf140b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.723+0000 7fdf1baa8700 1 -- 172.21.15.83:0/3071011860 shutdown_connections 2023-11-28T11:09:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.723+0000 7fdf1baa8700 1 -- 172.21.15.83:0/3071011860 wait complete. 2023-11-28T11:09:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.724+0000 7fdf1baa8700 1 Processor -- start 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.724+0000 7fdf1baa8700 1 -- start start 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.724+0000 7fdf1baa8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf1406c1b0 0x7fdf140ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 0x7fdf1414f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf1415c9c0 0x7fdf141f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141f2d20 con 0x7fdf140b24a0 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140ada00 con 0x7fdf1415c9c0 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140adce0 con 0x7fdf1406c1b0 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 0x7fdf1414f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 0x7fdf1414f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50360/0 (socket says 172.21.15.83:50360) 2023-11-28T11:09:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 -- 172.21.15.83:0/1397648979 learned_addr learned my addr 172.21.15.83:0/1397648979 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 -- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf1406c1b0 msgr2=0x7fdf140ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19844700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf1406c1b0 0x7fdf140ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1a045700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf1415c9c0 0x7fdf141f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf1406c1b0 0x7fdf140ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 -- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf1415c9c0 msgr2=0x7fdf141f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf1415c9c0 0x7fdf141f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 -- 172.21.15.83:0/1397648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040094a0 con 0x7fdf140b24a0 2023-11-28T11:09:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf19043700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 0x7fdf1414f9d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9b0 tx=0x7fdf1000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10009710 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1000c850 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1406b9b0 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000cbc0 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.725+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1406b6d0 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.726+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1406c630 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.727+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf10019030 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.727+0000 7fdf0a7fc700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdef80682f0 0x7fdef806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.727+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdf10085fd0 con 0x7fdf140b24a0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.727+0000 7fdf19844700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdef80682f0 0x7fdef806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.727+0000 7fdf19844700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdef80682f0 0x7fdef806a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf040090b0 tx=0x7fdf04008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.731+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdf10052500 con 0x7fdf140b24a0 2023-11-28T11:09:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.917+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf140ae330 con 0x7fdf140b24a0 2023-11-28T11:09:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.918+0000 7fdf0a7fc700 1 -- 172.21.15.83:0/1397648979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19737 (secure 0 0 0) 0x7fdf10021090 con 0x7fdf140b24a0 2023-11-28T11:09:33.920 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:33.920 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:25.004035+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:33.960379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:34.471584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:37.555175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:33.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdef80682f0 msgr2=0x7fdef806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdef80682f0 0x7fdef806a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf040090b0 tx=0x7fdf04008030 comp rx=0 tx=0).stop 2023-11-28T11:09:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 msgr2=0x7fdf1414f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 0x7fdf1414f9d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9b0 tx=0x7fdf1000ed70 comp rx=0 tx=0).stop 2023-11-28T11:09:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 shutdown_connections 2023-11-28T11:09:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf1415c9c0 0x7fdf141f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdf1406c1b0 0x7fdf140ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdef80682f0 0x7fdef806a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 --2- 172.21.15.83:0/1397648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf140b24a0 0x7fdf1414f9d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 >> 172.21.15.83:0/1397648979 conn(0x7fdf1406e9f0 msgr2=0x7fdf140b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 shutdown_connections 2023-11-28T11:09:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:33.920+0000 7fdf1baa8700 1 -- 172.21.15.83:0/1397648979 wait complete. 2023-11-28T11:09:33.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:09:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.055+0000 7f4770055700 1 Processor -- start 2023-11-28T11:09:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.055+0000 7f4770055700 1 -- start start 2023-11-28T11:09:34.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f4770055700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4768158ad0 0x7f47680ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f4770055700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f47680cd320 0x7f4768071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f4770055700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680cd940 con 0x7f4768071890 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f4770055700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4768072270 con 0x7f4768158ad0 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f4770055700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680cd740 con 0x7f47680cd320 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4768158ad0 0x7f47680ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4768158ad0 0x7f47680ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50366/0 (socket says 172.21.15.83:50366) 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 -- 172.21.15.83:0/3432076011 learned_addr learned my addr 172.21.15.83:0/3432076011 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 -- 172.21.15.83:0/3432076011 >> v2:172.21.15.90:3300/0 conn(0x7f47680cd320 msgr2=0x7f4768071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 --2- 172.21.15.83:0/3432076011 >> v2:172.21.15.90:3300/0 conn(0x7f47680cd320 0x7f4768071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 -- 172.21.15.83:0/3432076011 >> v1:172.21.15.90:6789/0 conn(0x7f4768071890 legacy=0x7f47680ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 -- 172.21.15.83:0/3432076011 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680cdc20 con 0x7f4768158ad0 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.056+0000 7f476ddf1700 1 --2- 172.21.15.83:0/3432076011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768158ad0 0x7f47680ce9e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4768070dd0 tx=0x7f475800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec170a86890fa2af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f476cdef700 1 -- 172.21.15.83:0/3432076011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475800dbe0 con 0x7f4768158ad0 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f476cdef700 1 -- 172.21.15.83:0/3432076011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475800dd40 con 0x7f4768158ad0 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f476cdef700 1 -- 172.21.15.83:0/3432076011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758004270 con 0x7f4768158ad0 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 -- 172.21.15.83:0/3432076011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768158ad0 msgr2=0x7f47680ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 --2- 172.21.15.83:0/3432076011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768158ad0 0x7f47680ce9e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4768070dd0 tx=0x7f475800be70 comp rx=0 tx=0).stop 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 -- 172.21.15.83:0/3432076011 shutdown_connections 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 --2- 172.21.15.83:0/3432076011 >> v2:172.21.15.90:3300/0 conn(0x7f47680cd320 0x7f4768071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 --2- 172.21.15.83:0/3432076011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768158ad0 0x7f47680ce9e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 -- 172.21.15.83:0/3432076011 >> 172.21.15.83:0/3432076011 conn(0x7f47680cba50 msgr2=0x7f47680cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 -- 172.21.15.83:0/3432076011 shutdown_connections 2023-11-28T11:09:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.057+0000 7f4770055700 1 -- 172.21.15.83:0/3432076011 wait complete. 2023-11-28T11:09:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.059+0000 7f4770055700 1 Processor -- start 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.059+0000 7f4770055700 1 -- start start 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f4770055700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 0x7f47681edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f4770055700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47680cd320 0x7f4768156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f4770055700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4768158ad0 0x7f47681f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f4770055700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681ee1f0 con 0x7f4768071890 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f4770055700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47680a0380 con 0x7f47680cd320 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f4770055700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47681ee420 con 0x7f4768158ad0 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 0x7f47681edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 0x7f47681edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50372/0 (socket says 172.21.15.83:50372) 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 -- 172.21.15.83:0/3866033532 learned_addr learned my addr 172.21.15.83:0/3866033532 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476d5f0700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47680cd320 0x7f4768156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476e5f2700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4768158ad0 0x7f47681f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 -- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4768158ad0 msgr2=0x7f47681f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4768158ad0 0x7f47681f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 -- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47680cd320 msgr2=0x7f4768156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47680cd320 0x7f4768156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 -- 172.21.15.83:0/3866033532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475800b7f0 con 0x7f4768071890 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.060+0000 7f476ddf1700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 0x7f47681edcb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f4758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.061+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580045f0 con 0x7f4768071890 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.061+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758022470 con 0x7f4768071890 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.061+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47680ce1e0 con 0x7f4768071890 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.061+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758022820 con 0x7f4768071890 2023-11-28T11:09:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.061+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47680cdf00 con 0x7f4768071890 2023-11-28T11:09:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.062+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47680ce4c0 con 0x7f4768071890 2023-11-28T11:09:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.062+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4758022980 con 0x7f4768071890 2023-11-28T11:09:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.062+0000 7f475e7fc700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f474c0682f0 0x7f474c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.063+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f475808a310 con 0x7f4768071890 2023-11-28T11:09:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.063+0000 7f476d5f0700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f474c0682f0 0x7f474c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.063+0000 7f476d5f0700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f474c0682f0 0x7f474c06a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f47680cc7e0 tx=0x7f47640090e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.067+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4758056840 con 0x7f4768071890 2023-11-28T11:09:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:34.291+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47680742b0 con 0x7f4768071890 2023-11-28T11:09:35.050 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-28T11:09:35.048+0000 7fdf5c1f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.056+0000 7f475e7fc700 1 -- 172.21.15.83:0/3866033532 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4758059e60 con 0x7f4768071890 2023-11-28T11:09:35.060 INFO:tasks.ceph.mds.c.smithi083.stderr: -6608> 2023-11-28T11:08:30.952+0000 7fce3cc2d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T11:09:35.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f474c0682f0 msgr2=0x7f474c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f474c0682f0 0x7f474c06a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f47680cc7e0 tx=0x7f47640090e0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 msgr2=0x7f47681edcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 0x7f47681edcb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f4758004750 comp rx=0 tx=0).stop 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 shutdown_connections 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4768158ad0 0x7f47681f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f474c0682f0 0x7f474c06a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4768071890 0x7f47681edcb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 --2- 172.21.15.83:0/3866033532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47680cd320 0x7f4768156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 >> 172.21.15.83:0/3866033532 conn(0x7f47680cba50 msgr2=0x7f47680c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 shutdown_connections 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.059+0000 7f4770055700 1 -- 172.21.15.83:0/3866033532 wait complete. 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:09:35.062 INFO:tasks.ceph.mds.c.smithi083.stderr: -5525> 2023-11-28T11:08:36.984+0000 7fce36c21700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:09:35.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:09:35.095 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-28T11:09:35.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.199+0000 7f3915f58700 1 Processor -- start 2023-11-28T11:09:35.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.199+0000 7f3915f58700 1 -- start start 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f3915f58700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f391006dc30 0x7f39100cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f3915f58700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39101601b0 0x7f39100cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f3915f58700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f39100cd670 0x7f391015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f3915f58700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391015cbc0 con 0x7f391006dc30 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f3915f58700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391015cdb0 con 0x7f39100cd670 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f3915f58700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3910160920 con 0x7f39101601b0 2023-11-28T11:09:35.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f391006dc30 0x7f39100cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f391006dc30 0x7f39100cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50392/0 (socket says 172.21.15.83:50392) 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 -- 172.21.15.83:0/1411395115 learned_addr learned my addr 172.21.15.83:0/1411395115 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390ffff700 1 --2- 172.21.15.83:0/1411395115 >> v2:172.21.15.90:3300/0 conn(0x7f39100cd670 0x7f391015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390effd700 1 --2- 172.21.15.83:0/1411395115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39101601b0 0x7f39100cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 -- 172.21.15.83:0/1411395115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39101601b0 msgr2=0x7f39100cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 --2- 172.21.15.83:0/1411395115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39101601b0 0x7f39100cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 -- 172.21.15.83:0/1411395115 >> v2:172.21.15.90:3300/0 conn(0x7f39100cd670 msgr2=0x7f391015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 --2- 172.21.15.83:0/1411395115 >> v2:172.21.15.90:3300/0 conn(0x7f39100cd670 0x7f391015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 -- 172.21.15.83:0/1411395115 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910160c00 con 0x7f391006dc30 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390ffff700 1 --2- 172.21.15.83:0/1411395115 >> v2:172.21.15.90:3300/0 conn(0x7f39100cd670 0x7f391015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.200+0000 7f390f7fe700 1 --2- 172.21.15.83:0/1411395115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391006dc30 0x7f39100cec30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39100ca560 tx=0x7f3904009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878e00cb59767244 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f390e7fc700 1 -- 172.21.15.83:0/1411395115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904005e30 con 0x7f391006dc30 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f390e7fc700 1 -- 172.21.15.83:0/1411395115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904004030 con 0x7f391006dc30 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f390e7fc700 1 -- 172.21.15.83:0/1411395115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39040043e0 con 0x7f391006dc30 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 -- 172.21.15.83:0/1411395115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391006dc30 msgr2=0x7f39100cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 --2- 172.21.15.83:0/1411395115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391006dc30 0x7f39100cec30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39100ca560 tx=0x7f3904009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 -- 172.21.15.83:0/1411395115 shutdown_connections 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 --2- 172.21.15.83:0/1411395115 >> v2:172.21.15.90:3300/0 conn(0x7f39100cd670 0x7f391015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 --2- 172.21.15.83:0/1411395115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391006dc30 0x7f39100cec30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 --2- 172.21.15.83:0/1411395115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39101601b0 0x7f39100cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 -- 172.21.15.83:0/1411395115 >> 172.21.15.83:0/1411395115 conn(0x7f39100b1bb0 msgr2=0x7f39100cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 -- 172.21.15.83:0/1411395115 shutdown_connections 2023-11-28T11:09:35.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.201+0000 7f3915f58700 1 -- 172.21.15.83:0/1411395115 wait complete. 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.202+0000 7f3915f58700 1 Processor -- start 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.202+0000 7f3915f58700 1 -- start start 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f3915f58700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f391006dc30 0x7f39101505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f3915f58700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39100cd670 0x7f39101f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f3915f58700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 0x7f39101599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f3915f58700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c4510 con 0x7f39101601b0 2023-11-28T11:09:35.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f3915f58700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3910150b10 con 0x7f391006dc30 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f3915f58700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c3470 con 0x7f39100cd670 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f390ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 0x7f39101599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f390ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 0x7f39101599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50402/0 (socket says 172.21.15.83:50402) 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f390ffff700 1 -- 172.21.15.83:0/2893208100 learned_addr learned my addr 172.21.15.83:0/2893208100 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f390f7fe700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f391006dc30 0x7f39101505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.203+0000 7f390effd700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39100cd670 0x7f39101f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f390ffff700 1 -- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39100cd670 msgr2=0x7f39101f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f390ffff700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39100cd670 0x7f39101f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f390ffff700 1 -- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f391006dc30 msgr2=0x7f39101505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f390ffff700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f391006dc30 0x7f39101505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f390ffff700 1 -- 172.21.15.83:0/2893208100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009500 con 0x7f39101601b0 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f390ffff700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 0x7f39101599f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3908007e10 tx=0x7f390800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800dcc0 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390800f040 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3910160600 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908003c60 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.204+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3910160ee0 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.205+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391006e000 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.205+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f390801c030 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.206+0000 7f38f7fff700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f38fc0682f0 0x7f38fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.206+0000 7f390f7fe700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f38fc0682f0 0x7f38fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.206+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f3908085e30 con 0x7f39101601b0 2023-11-28T11:09:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.206+0000 7f390f7fe700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f38fc0682f0 0x7f38fc06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f39100ca560 tx=0x7f3904008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.210+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3908052310 con 0x7f39101601b0 2023-11-28T11:09:35.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:35.436+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f39100aec00 con 0x7f39101601b0 2023-11-28T11:09:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.059+0000 7f38f7fff700 1 -- 172.21.15.83:0/2893208100 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3908055930 con 0x7f39101601b0 2023-11-28T11:09:36.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f38fc0682f0 msgr2=0x7f38fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f38fc0682f0 0x7f38fc06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f39100ca560 tx=0x7f3904008070 comp rx=0 tx=0).stop 2023-11-28T11:09:36.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 msgr2=0x7f39101599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 0x7f39101599f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3908007e10 tx=0x7f390800d3b0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 shutdown_connections 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f38fc0682f0 0x7f38fc06a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39101601b0 0x7f39101599f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f391006dc30 0x7f39101505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 --2- 172.21.15.83:0/2893208100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f39100cd670 0x7f39101f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 >> 172.21.15.83:0/2893208100 conn(0x7f39100b1bb0 msgr2=0x7f391006aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 shutdown_connections 2023-11-28T11:09:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.062+0000 7f3915f58700 1 -- 172.21.15.83:0/2893208100 wait complete. 2023-11-28T11:09:36.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:09:36.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.241+0000 7effa3bf8700 1 Processor -- start 2023-11-28T11:09:36.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.241+0000 7effa3bf8700 1 -- start start 2023-11-28T11:09:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa3bf8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff9c0cb520 0x7eff9c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa3bf8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c06cf30 0x7eff9c0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa3bf8700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c15d100 con 0x7eff9c15cd30 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa3bf8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c06beb0 con 0x7eff9c0cb520 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa3bf8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c06c170 con 0x7eff9c06cf30 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff9c0cb520 0x7eff9c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff9c0cb520 0x7eff9c0b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50410/0 (socket says 172.21.15.83:50410) 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 -- 172.21.15.83:0/840618683 learned_addr learned my addr 172.21.15.83:0/840618683 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa2195700 1 --2- 172.21.15.83:0/840618683 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c06cf30 0x7eff9c0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 -- 172.21.15.83:0/840618683 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c06cf30 msgr2=0x7eff9c0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 --2- 172.21.15.83:0/840618683 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c06cf30 0x7eff9c0b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 -- 172.21.15.83:0/840618683 >> v1:172.21.15.90:6789/0 conn(0x7eff9c15cd30 legacy=0x7eff9c0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 -- 172.21.15.83:0/840618683 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c06c450 con 0x7eff9c0cb520 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa1193700 1 --2- 172.21.15.83:0/840618683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c0cb520 0x7eff9c0b17e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0098f0 tx=0x7eff8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac328e8e25929cde server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.242+0000 7effa0992700 1 -- 172.21.15.83:0/840618683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00b8a0 con 0x7eff9c0cb520 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa0992700 1 -- 172.21.15.83:0/840618683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c00ba00 con 0x7eff9c0cb520 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa0992700 1 -- 172.21.15.83:0/840618683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00bd90 con 0x7eff9c0cb520 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 -- 172.21.15.83:0/840618683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c0cb520 msgr2=0x7eff9c0b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 --2- 172.21.15.83:0/840618683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c0cb520 0x7eff9c0b17e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0098f0 tx=0x7eff8c009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 -- 172.21.15.83:0/840618683 shutdown_connections 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 --2- 172.21.15.83:0/840618683 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c06cf30 0x7eff9c0b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 --2- 172.21.15.83:0/840618683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c0cb520 0x7eff9c0b17e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 -- 172.21.15.83:0/840618683 >> 172.21.15.83:0/840618683 conn(0x7eff9c0cc690 msgr2=0x7eff9c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 -- 172.21.15.83:0/840618683 shutdown_connections 2023-11-28T11:09:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.243+0000 7effa3bf8700 1 -- 172.21.15.83:0/840618683 wait complete. 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.244+0000 7effa3bf8700 1 Processor -- start 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.244+0000 7effa3bf8700 1 -- start start 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa3bf8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 0x7eff9c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa3bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff9c0cb520 0x7eff9c0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa3bf8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c15cd30 0x7eff9c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa3bf8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0ac010 con 0x7eff9c06cf30 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa3bf8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0aba50 con 0x7eff9c0cb520 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa3bf8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0abd30 con 0x7eff9c15cd30 2023-11-28T11:09:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 0x7eff9c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 0x7eff9c15a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50414/0 (socket says 172.21.15.83:50414) 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 -- 172.21.15.83:0/563419394 learned_addr learned my addr 172.21.15.83:0/563419394 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1193700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff9c0cb520 0x7eff9c0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa2195700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c15cd30 0x7eff9c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 -- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c15cd30 msgr2=0x7eff9c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c15cd30 0x7eff9c0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 -- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff9c0cb520 msgr2=0x7eff9c0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff9c0cb520 0x7eff9c0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.245+0000 7effa1994700 1 -- 172.21.15.83:0/563419394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c009550 con 0x7eff9c06cf30 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.246+0000 7effa1994700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 0x7eff9c15a680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c450 tx=0x7eff9800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.246+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980052d0 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.246+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98005430 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.246+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980038c0 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.246+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c06ca10 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.246+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c06c730 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.247+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c0b2260 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.247+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff98019030 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.248+0000 7eff927fc700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff800682f0 0x7eff8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.248+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7eff98085130 con 0x7eff9c06cf30 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.248+0000 7effa1193700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff800682f0 0x7eff8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.248+0000 7effa1193700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff800682f0 0x7eff8006a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0098f0 tx=0x7eff8c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.252+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff98051520 con 0x7eff9c06cf30 2023-11-28T11:09:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:36.435+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff9c15d100 con 0x7eff9c06cf30 2023-11-28T11:09:37.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.116+0000 7eff927fc700 1 -- 172.21.15.83:0/563419394 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7eff9c15d100 con 0x7eff9c06cf30 2023-11-28T11:09:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff800682f0 msgr2=0x7eff8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff800682f0 0x7eff8006a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0098f0 tx=0x7eff8c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 msgr2=0x7eff9c15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 0x7eff9c15a680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c450 tx=0x7eff9800c760 comp rx=0 tx=0).stop 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 shutdown_connections 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff9c15cd30 0x7eff9c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff800682f0 0x7eff8006a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9c06cf30 0x7eff9c15a680 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 --2- 172.21.15.83:0/563419394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff9c0cb520 0x7eff9c0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 >> 172.21.15.83:0/563419394 conn(0x7eff9c0cc690 msgr2=0x7eff9c0b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 shutdown_connections 2023-11-28T11:09:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.119+0000 7effa3bf8700 1 -- 172.21.15.83:0/563419394 wait complete. 2023-11-28T11:09:37.122 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:09:37.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:09:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.264+0000 7f7bd32b7700 1 Processor -- start 2023-11-28T11:09:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.264+0000 7f7bd32b7700 1 -- start start 2023-11-28T11:09:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd32b7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 0x7f7bcc0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd32b7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc15d100 con 0x7f7bcc06cf30 2023-11-28T11:09:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd32b7700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc06beb0 con 0x7f7bcc0b1ef0 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd32b7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b2330 con 0x7f7bcc15cd30 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd0852700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7bcc06cf30 0x7f7bcc0b17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48282/0 (socket says 172.21.15.83:48282) 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd0852700 1 -- 172.21.15.83:0/2140050447 learned_addr learned my addr 172.21.15.83:0/2140050447 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428404950 0 0) 0x7f7bbc001110 con 0x7f7bcc06cf30 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 0x7f7bcc0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb80035b0 con 0x7f7bcc06cf30 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303826291 0 0) 0x7f7bb80035b0 con 0x7f7bcc06cf30 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 msgr2=0x7f7bcc0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 0x7f7bcc0cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 >> v1:172.21.15.90:6789/0 conn(0x7f7bcc0b1ef0 legacy=0x7f7bcc0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc06c170 con 0x7f7bcc06cf30 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.265+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f7bbc001800 con 0x7f7bcc06cf30 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 >> v1:172.21.15.83:6789/0 conn(0x7f7bcc06cf30 legacy=0x7f7bcc0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb8003a30 0x7f7bb8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8004850 0x7f7bb8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 0x7f7bb8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b2330 con 0x7f7bb8003a30 2023-11-28T11:09:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800bac0 con 0x7f7bb8004850 2023-11-28T11:09:37.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800bc50 con 0x7f7bcc15cd30 2023-11-28T11:09:37.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb8003a30 0x7f7bb8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd0852700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8004850 0x7f7bb8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1854700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 0x7f7bb8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 -- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 msgr2=0x7f7bb8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc15cd30 0x7f7bb8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 -- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8004850 msgr2=0x7f7bb8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8004850 0x7f7bb8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 -- 172.21.15.83:0/2140050447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80035b0 con 0x7f7bb8003a30 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb8003a30 0x7f7bb8003e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009b60 tx=0x7f7bb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4536540acedfbe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800e6c0 con 0x7f7bb8003a30 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800e820 con 0x7f7bb8003a30 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.266+0000 7f7bc3fff700 1 -- 172.21.15.83:0/2140050447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800eb90 con 0x7f7bb8003a30 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2140050447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb8003a30 msgr2=0x7f7bb8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2140050447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb8003a30 0x7f7bb8003e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009b60 tx=0x7f7bb8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:37.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd1854700 1 -- 172.21.15.83:0/2140050447 reap_dead start 2023-11-28T11:09:37.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2140050447 shutdown_connections 2023-11-28T11:09:37.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2140050447 >> 172.21.15.83:0/2140050447 conn(0x7f7bcc0cc690 msgr2=0x7f7bcc0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2140050447 shutdown_connections 2023-11-28T11:09:37.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.267+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2140050447 wait complete. 2023-11-28T11:09:37.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.268+0000 7f7bd32b7700 1 Processor -- start 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.268+0000 7f7bd32b7700 1 -- start start 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd32b7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8003a30 0x7f7bcc148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd32b7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc0c7ae0 0x7f7bcc0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd32b7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 0x7f7bcc0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd32b7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cb090 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd32b7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc148570 con 0x7f7bb8003a30 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd32b7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c9b20 con 0x7f7bcc0c7ae0 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 0x7f7bcc0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 0x7f7bcc0cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50436/0 (socket says 172.21.15.83:50436) 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 -- 172.21.15.83:0/2544725286 learned_addr learned my addr 172.21.15.83:0/2544725286 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8003a30 0x7f7bcc148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd0852700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc0c7ae0 0x7f7bcc0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 -- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc0c7ae0 msgr2=0x7f7bcc0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc0c7ae0 0x7f7bcc0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 -- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8003a30 msgr2=0x7f7bcc148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8003a30 0x7f7bcc148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 -- 172.21.15.83:0/2544725286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc06c730 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.269+0000 7f7bd1854700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 0x7f7bcc0cab50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc071630 tx=0x7f7bc400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.270+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4003c10 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.270+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc4003d90 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.270+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc06c450 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.270+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400da10 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.270+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc06ca10 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.271+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc0742b0 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.271+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7bc400db70 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.271+0000 7f7bc1ffb700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7bb00682f0 0x7f7bb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.271+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f7bc4086140 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.271+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7bb00682f0 0x7f7bb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.272+0000 7f7bd1053700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7bb00682f0 0x7f7bb006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80090b0 tx=0x7f7bc801f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.276+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7bc40527d0 con 0x7f7bcc0ca6b0 2023-11-28T11:09:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:37.458+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7bcc0aff50 con 0x7f7bcc0ca6b0 2023-11-28T11:09:38.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.121+0000 7f7bc1ffb700 1 -- 172.21.15.83:0/2544725286 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7bc4055df0 con 0x7f7bcc0ca6b0 2023-11-28T11:09:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.129+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7bb00682f0 msgr2=0x7f7bb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.129+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7bb00682f0 0x7f7bb006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80090b0 tx=0x7f7bc801f040 comp rx=0 tx=0).stop 2023-11-28T11:09:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.129+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 msgr2=0x7f7bcc0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.129+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 0x7f7bcc0cab50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc071630 tx=0x7f7bc400d150 comp rx=0 tx=0).stop 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 shutdown_connections 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7bb00682f0 0x7f7bb006a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bcc0ca6b0 0x7f7bcc0cab50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bb8003a30 0x7f7bcc148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 --2- 172.21.15.83:0/2544725286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bcc0c7ae0 0x7f7bcc0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 >> 172.21.15.83:0/2544725286 conn(0x7f7bcc0cc690 msgr2=0x7f7bcc0b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 shutdown_connections 2023-11-28T11:09:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.130+0000 7f7bd32b7700 1 -- 172.21.15.83:0/2544725286 wait complete. 2023-11-28T11:09:38.133 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-28T11:09:38.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-28T11:09:38.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.276+0000 7efe09832700 1 Processor -- start 2023-11-28T11:09:38.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.276+0000 7efe09832700 1 -- start start 2023-11-28T11:09:38.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe09832700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7efe0415cd30 0x7efe040cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe09832700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe0406cf30 0x7efe040cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe09832700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe09832700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0415d100 con 0x7efe0406cf30 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe09832700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0406beb0 con 0x7efe0415cd30 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe09832700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0406c170 con 0x7efe040b17e0 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe027fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe0406cf30 0x7efe040cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe027fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe0406cf30 0x7efe040cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50442/0 (socket says 172.21.15.83:50442) 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.277+0000 7efe027fc700 1 -- 172.21.15.83:0/3002359801 learned_addr learned my addr 172.21.15.83:0/3002359801 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe02ffd700 1 --2- 172.21.15.83:0/3002359801 >> v2:172.21.15.90:3300/0 conn(0x7efe0415cd30 0x7efe040cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe037fe700 1 --2- 172.21.15.83:0/3002359801 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe027fc700 1 -- 172.21.15.83:0/3002359801 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 msgr2=0x7efe040b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe027fc700 1 --2- 172.21.15.83:0/3002359801 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe027fc700 1 -- 172.21.15.83:0/3002359801 >> v2:172.21.15.90:3300/0 conn(0x7efe0415cd30 msgr2=0x7efe040cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe027fc700 1 --2- 172.21.15.83:0/3002359801 >> v2:172.21.15.90:3300/0 conn(0x7efe0415cd30 0x7efe040cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe027fc700 1 -- 172.21.15.83:0/3002359801 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0406c450 con 0x7efe0406cf30 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe02ffd700 1 --2- 172.21.15.83:0/3002359801 >> v2:172.21.15.90:3300/0 conn(0x7efe0415cd30 0x7efe040cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe027fc700 1 --2- 172.21.15.83:0/3002359801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe040cb350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe04071630 tx=0x7efdf4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f30247e86cebbc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe01ffb700 1 -- 172.21.15.83:0/3002359801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400b870 con 0x7efe0406cf30 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe01ffb700 1 -- 172.21.15.83:0/3002359801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400b9d0 con 0x7efe0406cf30 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe01ffb700 1 -- 172.21.15.83:0/3002359801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400bd60 con 0x7efe0406cf30 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe09832700 1 -- 172.21.15.83:0/3002359801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 msgr2=0x7efe040cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.278+0000 7efe09832700 1 --2- 172.21.15.83:0/3002359801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe040cb350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe04071630 tx=0x7efdf4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 -- 172.21.15.83:0/3002359801 shutdown_connections 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 --2- 172.21.15.83:0/3002359801 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 --2- 172.21.15.83:0/3002359801 >> v2:172.21.15.90:3300/0 conn(0x7efe0415cd30 0x7efe040cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 --2- 172.21.15.83:0/3002359801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe040cb350 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 -- 172.21.15.83:0/3002359801 >> 172.21.15.83:0/3002359801 conn(0x7efe040cc690 msgr2=0x7efe040cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 -- 172.21.15.83:0/3002359801 shutdown_connections 2023-11-28T11:09:38.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.279+0000 7efe09832700 1 -- 172.21.15.83:0/3002359801 wait complete. 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.280+0000 7efe09832700 1 Processor -- start 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.280+0000 7efe09832700 1 -- start start 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe0415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0415cd30 0x7efe040add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c6ae0 con 0x7efe0406cf30 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c62d0 con 0x7efe0415cd30 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c6520 con 0x7efe040b17e0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe0415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe0415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50448/0 (socket says 172.21.15.83:50448) 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 -- 172.21.15.83:0/798529644 learned_addr learned my addr 172.21.15.83:0/798529644 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe037fe700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0415cd30 0x7efe040add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe027fc700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 -- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 msgr2=0x7efe040ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 -- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0415cd30 msgr2=0x7efe040add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0415cd30 0x7efe040add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 -- 172.21.15.83:0/798529644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009550 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe02ffd700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe0415ad80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c7f0 tx=0x7efdf800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8011800 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8011960 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8011d10 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0406ca10 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.281+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0406c730 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.282+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe040b2160 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.283+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efdf8021070 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.283+0000 7efdebfff700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdf00682f0 0x7efdf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.283+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6047+0+0 (secure 0 0 0) 0x7efdf8085b00 con 0x7efe0406cf30 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.283+0000 7efe027fc700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdf00682f0 0x7efdf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.283+0000 7efe027fc700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdf00682f0 0x7efdf006a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe04071630 tx=0x7efdf4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.287+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efdf8052370 con 0x7efe0406cf30 2023-11-28T11:09:38.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:38.468+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efe040742b0 con 0x7efe0406cf30 2023-11-28T11:09:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.122+0000 7efdebfff700 1 -- 172.21.15.83:0/798529644 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7efdf8055990 con 0x7efe0406cf30 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdf00682f0 msgr2=0x7efdf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdf00682f0 0x7efdf006a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe04071630 tx=0x7efdf4008030 comp rx=0 tx=0).stop 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 msgr2=0x7efe0415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe0415ad80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c7f0 tx=0x7efdf800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 shutdown_connections 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0415cd30 0x7efe040add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efdf00682f0 0x7efdf006a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe0406cf30 0x7efe0415ad80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 --2- 172.21.15.83:0/798529644 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efe040b17e0 0x7efe040ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 >> 172.21.15.83:0/798529644 conn(0x7efe040cc690 msgr2=0x7efe040b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.126+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 shutdown_connections 2023-11-28T11:09:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.127+0000 7efe09832700 1 -- 172.21.15.83:0/798529644 wait complete. 2023-11-28T11:09:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data_ec' removed 2023-11-28T11:09:39.143 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:09:39.143 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:09:39.143 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:09:39.143 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:39.144 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:09:39.144 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:09:39.144 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:09:39.144 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:39.144 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:09:39.144 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:09:39.145 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:09:39.145 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:39.145 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:09:39.145 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:09:39.145 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:09:39.145 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:39.153 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:09:39.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:09:39.154 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:09:39.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:09:39.155 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:09:39.156 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:09:39.157 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:09:39.157 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:09:39.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:39.195 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:09:39.196 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:39.197 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:09:39.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:39.198 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:39.199 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:09:39.199 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:09:39.200 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:39.200 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:39.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:09:39.276 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-28T11:09:39.277 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-28T11:09:39.289 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2023-11-28T11:09:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 Processor -- start 2023-11-28T11:09:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 -- start start 2023-11-28T11:09:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6e94158040 0x7f6e9415b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e940d0560 0x7f6e940ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e940cd4f0 0x7f6e940b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e940cea60 con 0x7f6e940cd4f0 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e940b3660 con 0x7f6e94158040 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.369+0000 7f6e9ae89700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9415ba90 con 0x7f6e940d0560 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e940cd4f0 0x7f6e940b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e940cd4f0 0x7f6e940b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50490/0 (socket says 172.21.15.83:50490) 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 -- 172.21.15.83:0/580993968 learned_addr learned my addr 172.21.15.83:0/580993968 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e98c25700 1 --2- 172.21.15.83:0/580993968 >> v2:172.21.15.90:3300/0 conn(0x7f6e94158040 0x7f6e9415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 -- 172.21.15.83:0/580993968 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e940d0560 msgr2=0x7f6e940ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e93fff700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e940d0560 0x7f6e940ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e940d0560 0x7f6e940ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 -- 172.21.15.83:0/580993968 >> v2:172.21.15.90:3300/0 conn(0x7f6e94158040 msgr2=0x7f6e9415b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 --2- 172.21.15.83:0/580993968 >> v2:172.21.15.90:3300/0 conn(0x7f6e94158040 0x7f6e9415b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 -- 172.21.15.83:0/580993968 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940d0a00 con 0x7f6e940cd4f0 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e98c25700 1 --2- 172.21.15.83:0/580993968 >> v2:172.21.15.90:3300/0 conn(0x7f6e94158040 0x7f6e9415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e93fff700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e940d0560 0x7f6e940ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e99426700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940cd4f0 0x7f6e940b27e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e940d0cc0 tx=0x7f6e8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bf22f54224c8fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e937fe700 1 -- 172.21.15.83:0/580993968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00e070 con 0x7f6e940cd4f0 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e937fe700 1 -- 172.21.15.83:0/580993968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c00b9c0 con 0x7f6e940cd4f0 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.370+0000 7f6e937fe700 1 -- 172.21.15.83:0/580993968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00bd70 con 0x7f6e940cd4f0 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 -- 172.21.15.83:0/580993968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940cd4f0 msgr2=0x7f6e940b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940cd4f0 0x7f6e940b27e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e940d0cc0 tx=0x7f6e8c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 -- 172.21.15.83:0/580993968 shutdown_connections 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940cd4f0 0x7f6e940b27e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/580993968 >> v2:172.21.15.90:3300/0 conn(0x7f6e94158040 0x7f6e9415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/580993968 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e940d0560 0x7f6e940ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 -- 172.21.15.83:0/580993968 >> 172.21.15.83:0/580993968 conn(0x7f6e9406de20 msgr2=0x7f6e9406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 -- 172.21.15.83:0/580993968 shutdown_connections 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.371+0000 7f6e9ae89700 1 -- 172.21.15.83:0/580993968 wait complete. 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.372+0000 7f6e9ae89700 1 Processor -- start 2023-11-28T11:09:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.372+0000 7f6e9ae89700 1 -- start start 2023-11-28T11:09:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e9ae89700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940cd4f0 0x7f6e9414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e9ae89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 0x7f6e940b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e9ae89700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e94158040 0x7f6e94155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e9ae89700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940c4a10 con 0x7f6e940d0560 2023-11-28T11:09:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e9ae89700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940c4cf0 con 0x7f6e940cd4f0 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e9ae89700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940c4fd0 con 0x7f6e94158040 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 0x7f6e940b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 0x7f6e940b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50494/0 (socket says 172.21.15.83:50494) 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 -- 172.21.15.83:0/2399157690 learned_addr learned my addr 172.21.15.83:0/2399157690 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e98c25700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940cd4f0 0x7f6e9414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e99426700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e94158040 0x7f6e94155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 -- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e94158040 msgr2=0x7f6e94155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e94158040 0x7f6e94155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 -- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940cd4f0 msgr2=0x7f6e9414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940cd4f0 0x7f6e9414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 -- 172.21.15.83:0/2399157690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c009b30 con 0x7f6e940d0560 2023-11-28T11:09:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e93fff700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 0x7f6e940b3b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c7c0 tx=0x7f6e8000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011800 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80011960 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.373+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011cf0 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.374+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e940cb730 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.374+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e940cb450 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.374+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e94158410 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.375+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e80021070 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.375+0000 7f6e917fa700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.375+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6e800857e0 con 0x7f6e940d0560 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.375+0000 7f6e98c25700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.376+0000 7f6e98c25700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e88006fd0 tx=0x7f6e88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.379+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e80052200 con 0x7f6e940d0560 2023-11-28T11:09:39.404 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2023-11-28T11:09:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:39.584+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6e940c4e30 con 0x7f6e940d0560 2023-11-28T11:09:40.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.430+0000 7f6e917fa700 1 -- 172.21.15.83:0/2399157690 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6e940c4e30 con 0x7f6e940d0560 2023-11-28T11:09:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 msgr2=0x7f6e7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e88006fd0 tx=0x7f6e88008040 comp rx=0 tx=0).stop 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 msgr2=0x7f6e940b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 0x7f6e940b3b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c7c0 tx=0x7f6e8000cad0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 shutdown_connections 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e94158040 0x7f6e94155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940cd4f0 0x7f6e9414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 --2- 172.21.15.83:0/2399157690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e940d0560 0x7f6e940b3b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 >> 172.21.15.83:0/2399157690 conn(0x7f6e9406de20 msgr2=0x7f6e940b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 shutdown_connections 2023-11-28T11:09:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.433+0000 7f6e9ae89700 1 -- 172.21.15.83:0/2399157690 wait complete. 2023-11-28T11:09:40.436 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-28T11:09:40.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:09:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.609+0000 7f1c78c11700 1 Processor -- start 2023-11-28T11:09:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.609+0000 7f1c78c11700 1 -- start start 2023-11-28T11:09:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.610+0000 7f1c78c11700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c7415a510 0x7f1c740cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.610+0000 7f1c78c11700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1c7415a8e0 0x7f1c740b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.610+0000 7f1c78c11700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7415f1d0 con 0x7f1c740cfdf0 2023-11-28T11:09:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.610+0000 7f1c78c11700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740cd780 con 0x7f1c7415a510 2023-11-28T11:09:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.610+0000 7f1c78c11700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7415eef0 con 0x7f1c7415a8e0 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c7415a510 0x7f1c740cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c71d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1c740cfdf0 0x7f1c740b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48366/0 (socket says 172.21.15.83:48366) 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c71d9b700 1 -- 172.21.15.83:0/1454160035 learned_addr learned my addr 172.21.15.83:0/1454160035 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c72d9d700 1 --2- 172.21.15.83:0/1454160035 >> v2:172.21.15.90:3300/0 conn(0x7f1c7415a8e0 0x7f1c740b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7259c700 1 -- 172.21.15.83:0/1454160035 >> v2:172.21.15.90:3300/0 conn(0x7f1c7415a8e0 msgr2=0x7f1c740b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7259c700 1 --2- 172.21.15.83:0/1454160035 >> v2:172.21.15.90:3300/0 conn(0x7f1c7415a8e0 0x7f1c740b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7259c700 1 -- 172.21.15.83:0/1454160035 >> v1:172.21.15.83:6789/0 conn(0x7f1c740cfdf0 legacy=0x7f1c740b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7159a700 1 -- 172.21.15.83:0/1454160035 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909125950 0 0) 0x7f1c7415f1d0 con 0x7f1c740cfdf0 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7259c700 1 -- 172.21.15.83:0/1454160035 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740cedd0 con 0x7f1c7415a510 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7259c700 1 --2- 172.21.15.83:0/1454160035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a510 0x7f1c740cf8b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1c740718c0 tx=0x7f1c68009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58ab999f7174284 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7159a700 1 -- 172.21.15.83:0/1454160035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011830 con 0x7f1c7415a510 2023-11-28T11:09:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7159a700 1 -- 172.21.15.83:0/1454160035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68011990 con 0x7f1c7415a510 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.611+0000 7f1c7159a700 1 -- 172.21.15.83:0/1454160035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011d20 con 0x7f1c7415a510 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 -- 172.21.15.83:0/1454160035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a510 msgr2=0x7f1c740cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 --2- 172.21.15.83:0/1454160035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a510 0x7f1c740cf8b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1c740718c0 tx=0x7f1c68009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 -- 172.21.15.83:0/1454160035 shutdown_connections 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 --2- 172.21.15.83:0/1454160035 >> v2:172.21.15.90:3300/0 conn(0x7f1c7415a8e0 0x7f1c740b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 --2- 172.21.15.83:0/1454160035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a510 0x7f1c740cf8b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 -- 172.21.15.83:0/1454160035 >> 172.21.15.83:0/1454160035 conn(0x7f1c740cd960 msgr2=0x7f1c740cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 -- 172.21.15.83:0/1454160035 shutdown_connections 2023-11-28T11:09:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.612+0000 7f1c78c11700 1 -- 172.21.15.83:0/1454160035 wait complete. 2023-11-28T11:09:40.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.613+0000 7f1c78c11700 1 Processor -- start 2023-11-28T11:09:40.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.613+0000 7f1c78c11700 1 -- start start 2023-11-28T11:09:40.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c78c11700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c740cfdf0 0x7f1c740c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c78c11700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7415a510 0x7f1c740c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c78c11700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 0x7f1c740c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c78c11700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740ca020 con 0x7f1c7415a8e0 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c78c11700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c6300 con 0x7f1c740cfdf0 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c78c11700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c74149c70 con 0x7f1c7415a510 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 0x7f1c740c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 0x7f1c740c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50514/0 (socket says 172.21.15.83:50514) 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 -- 172.21.15.83:0/2080627970 learned_addr learned my addr 172.21.15.83:0/2080627970 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c71d9b700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7415a510 0x7f1c740c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c7259c700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c740cfdf0 0x7f1c740c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 -- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7415a510 msgr2=0x7f1c740c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7415a510 0x7f1c740c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 -- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c740cfdf0 msgr2=0x7f1c740c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c740cfdf0 0x7f1c740c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 -- 172.21.15.83:0/2080627970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c680094a0 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.614+0000 7f1c72d9d700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 0x7f1c740c4fb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00cda0 tx=0x7f1c6c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.615+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c003a20 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.615+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c003b80 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.615+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740cf390 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.615+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004020 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.615+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740cf0b0 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.615+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c740b3070 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.616+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c6c01c020 con 0x7f1c7415a8e0 2023-11-28T11:09:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.616+0000 7f1c62ffd700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c540682f0 0x7f1c5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.616+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1c6c012070 con 0x7f1c7415a8e0 2023-11-28T11:09:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.616+0000 7f1c7259c700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c540682f0 0x7f1c5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.617+0000 7f1c7259c700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c540682f0 0x7f1c5406a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c680090b0 tx=0x7f1c68008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.621+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c6c052b10 con 0x7f1c7415a8e0 2023-11-28T11:09:40.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.874+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1c740c54f0 con 0x7f1c7415a8e0 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.874+0000 7f1c62ffd700 1 -- 172.21.15.83:0/2080627970 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f1c6c056130 con 0x7f1c7415a8e0 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-28T11:09:40.876 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVly5GSMRAArlawRsTlzQ8deoXsPnYmsg==", 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVl2bkZAhAA15xpS3O6vPGdnCqW98I/0w==", 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.878 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlCEbQOBAA8FKGXdCurT68yNjSUEJigQ==", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlQFxUCRAA9BmlOX7cM5ewNextI+L6lg==", 2023-11-28T11:09:40.879 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAAymVlMHf8JRAAqxuxtuaslP93VXhK17KfXA==", 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQADymVlUrMkJRAAxCUtzDNzfHRTJ4jLzeTa0Q==", 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAGymVlZqxOJBAAzhcB9ERpDVy5FCZNn1uMLg==", 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAJymVlPZEhJBAAm8fhc9fRDDq3ZROV4Mez1Q==", 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.883 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAPymVlqwREKBAAsRlS/3ztcuWSZ9nRXZDKuA==", 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQASymVlQKsoKBAABR2d0tZ/TLRKIDyQrAKVkA==", 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAZymVlqduLAhAAPau0X+UZigxTAoWpmUnzRg==", 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.885 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAcymVlckzLBBAAH4vnm0MWED/F40ZWq2uIog==", 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.886 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVllJ2YDRAAE9HeK1zyWWnOIRh09fAG3Q==", 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVln8eKFhAAPGbMQ2n6PsaH6hEURGRJvw==", 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.888 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlXJRCExAATH4e/FL5+nvYhdsZ7mB9LQ==", 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-28T11:09:40.889 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlo7c6HBAAHpELpEY29uzkqcJ+jAaxRA==", 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlC9hgFRAAL7ZVG6EzaROjM+to9qi5ZQ==", 2023-11-28T11:09:40.890 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mds", 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAiymVlF1ecAhAAd6GpFv6aHRrlwp2YbWuJTw==", 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T11:09:40.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAiymVl+V+cAhAABI/VerHuLHnJ88LfZXYtlg==", 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-osd", 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAiymVlq2mcAhAATVhzy0YUfUCeLuXa271Jvg==", 2023-11-28T11:09:40.892 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAiymVlrXGcAhAAUkqm+Klr2g1J75jwDpjYXg==", 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAiymVlfXmcAhAAYedhtSAPtZTduQ3A7O2qCw==", 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAiymVlbYGcAhAAo+ylVWDZYrYq2lePl6WdYg==", 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T11:09:40.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlBfMxLhAA6N4Cnno8jNhgzS5eYJKWqQ==", 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:40.895 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlHSGFKhAA3pr4QAikKKmqCfS0orTyWw==", 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-28T11:09:40.896 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:09:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c540682f0 msgr2=0x7f1c5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c540682f0 0x7f1c5406a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c680090b0 tx=0x7f1c68008030 comp rx=0 tx=0).stop 2023-11-28T11:09:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 msgr2=0x7f1c740c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 0x7f1c740c4fb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00cda0 tx=0x7f1c6c00a570 comp rx=0 tx=0).stop 2023-11-28T11:09:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 shutdown_connections 2023-11-28T11:09:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c540682f0 0x7f1c5406a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c7415a8e0 0x7f1c740c4fb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c740cfdf0 0x7f1c740c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 --2- 172.21.15.83:0/2080627970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7415a510 0x7f1c740c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 >> 172.21.15.83:0/2080627970 conn(0x7f1c740cd960 msgr2=0x7f1c740b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 shutdown_connections 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:40.876+0000 7f1c78c11700 1 -- 172.21.15.83:0/2080627970 wait complete. 2023-11-28T11:09:40.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T11:09:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.019+0000 7f17afc0d700 1 Processor -- start 2023-11-28T11:09:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.019+0000 7f17afc0d700 1 -- start start 2023-11-28T11:09:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17afc0d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a815e5c0 0x7f17a815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17afc0d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f17a815b3b0 0x7f17a80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17afc0d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17a815b820 0x7f17a80d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17afc0d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a8151a30 con 0x7f17a815b820 2023-11-28T11:09:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17afc0d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80cb460 con 0x7f17a815b3b0 2023-11-28T11:09:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17afc0d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80cb740 con 0x7f17a815e5c0 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17a815b820 0x7f17a80d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17a815b820 0x7f17a80d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50522/0 (socket says 172.21.15.83:50522) 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 -- 172.21.15.83:0/3885338064 learned_addr learned my addr 172.21.15.83:0/3885338064 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ad9a9700 1 --2- 172.21.15.83:0/3885338064 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a815e5c0 0x7f17a815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ad1a8700 1 --2- 172.21.15.83:0/3885338064 >> v2:172.21.15.90:3300/0 conn(0x7f17a815b3b0 0x7f17a80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 -- 172.21.15.83:0/3885338064 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a815e5c0 msgr2=0x7f17a815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 --2- 172.21.15.83:0/3885338064 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a815e5c0 0x7f17a815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 -- 172.21.15.83:0/3885338064 >> v2:172.21.15.90:3300/0 conn(0x7f17a815b3b0 msgr2=0x7f17a80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 --2- 172.21.15.83:0/3885338064 >> v2:172.21.15.90:3300/0 conn(0x7f17a815b3b0 0x7f17a80d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 -- 172.21.15.83:0/3885338064 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a815abb0 con 0x7f17a815b820 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ae1aa700 1 --2- 172.21.15.83:0/3885338064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815b820 0x7f17a80d1020 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f17a80ce7c0 tx=0x7f17a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874451968f81ef4c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ac9a7700 1 -- 172.21.15.83:0/3885338064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000e070 con 0x7f17a815b820 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ac9a7700 1 -- 172.21.15.83:0/3885338064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a000b9c0 con 0x7f17a815b820 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.020+0000 7f17ac9a7700 1 -- 172.21.15.83:0/3885338064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000bd70 con 0x7f17a815b820 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 -- 172.21.15.83:0/3885338064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815b820 msgr2=0x7f17a80d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3885338064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815b820 0x7f17a80d1020 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f17a80ce7c0 tx=0x7f17a000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 -- 172.21.15.83:0/3885338064 shutdown_connections 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3885338064 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a815e5c0 0x7f17a815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3885338064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815b820 0x7f17a80d1020 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3885338064 >> v2:172.21.15.90:3300/0 conn(0x7f17a815b3b0 0x7f17a80d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 -- 172.21.15.83:0/3885338064 >> 172.21.15.83:0/3885338064 conn(0x7f17a806ea50 msgr2=0x7f17a8070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 -- 172.21.15.83:0/3885338064 shutdown_connections 2023-11-28T11:09:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.021+0000 7f17afc0d700 1 -- 172.21.15.83:0/3885338064 wait complete. 2023-11-28T11:09:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.023+0000 7f17afc0d700 1 Processor -- start 2023-11-28T11:09:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.023+0000 7f17afc0d700 1 -- start start 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a815b820 0x7f17a81460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 0x7f17a80c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a80c47e0 0x7f17a80c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80b49a0 con 0x7f17a815e5c0 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80b1ed0 con 0x7f17a815b820 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80b21b0 con 0x7f17a80c47e0 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 0x7f17a80c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 0x7f17a80c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50526/0 (socket says 172.21.15.83:50526) 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 -- 172.21.15.83:0/3571104979 learned_addr learned my addr 172.21.15.83:0/3571104979 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 -- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a80c47e0 msgr2=0x7f17a80c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a80c47e0 0x7f17a80c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 -- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a815b820 msgr2=0x7f17a81460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad9a9700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a815b820 0x7f17a81460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a815b820 0x7f17a81460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 -- 172.21.15.83:0/3571104979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0009b30 con 0x7f17a815e5c0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17ad1a8700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 0x7f17a80c7060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f179800d800 tx=0x7f179800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17980125f0 con 0x7f17a815e5c0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1798012750 con 0x7f17a815e5c0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798012ae0 con 0x7f17a815e5c0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a815eed0 con 0x7f17a815e5c0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.024+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a815ae90 con 0x7f17a815e5c0 2023-11-28T11:09:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.025+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a80cba20 con 0x7f17a815e5c0 2023-11-28T11:09:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.026+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1798012c40 con 0x7f17a815e5c0 2023-11-28T11:09:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.026+0000 7f179e7fc700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f178c0682f0 0x7f178c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.026+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1798085f20 con 0x7f17a815e5c0 2023-11-28T11:09:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.026+0000 7f17ad9a9700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f178c0682f0 0x7f178c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.027+0000 7f17ad9a9700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f178c0682f0 0x7f178c06a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009950 tx=0x7f17a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.030+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1798052990 con 0x7f17a815e5c0 2023-11-28T11:09:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.288+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f17a80742b0 con 0x7f17a815e5c0 2023-11-28T11:09:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.291+0000 7f179e7fc700 1 -- 172.21.15.83:0/3571104979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1798012ed0 con 0x7f17a815e5c0 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f178c0682f0 msgr2=0x7f178c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f178c0682f0 0x7f178c06a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009950 tx=0x7f17a4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 msgr2=0x7f17a80c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 0x7f17a80c7060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f179800d800 tx=0x7f179800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 shutdown_connections 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f17a80c47e0 0x7f17a80c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a815b820 0x7f17a81460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f178c0682f0 0x7f178c06a7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 --2- 172.21.15.83:0/3571104979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a815e5c0 0x7f17a80c7060 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 >> 172.21.15.83:0/3571104979 conn(0x7f17a815b3b0 msgr2=0x7f17a80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 shutdown_connections 2023-11-28T11:09:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.293+0000 7f17afc0d700 1 -- 172.21.15.83:0/3571104979 wait complete. 2023-11-28T11:09:41.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T11:09:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.426+0000 7fb9635ff700 1 Processor -- start 2023-11-28T11:09:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.426+0000 7fb9635ff700 1 -- start start 2023-11-28T11:09:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb9635ff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb95c0b1900 0x7fb95c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb9635ff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb95c15e000 0x7fb95c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb9635ff700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c06bc10 con 0x7fb95c0b5600 2023-11-28T11:09:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb9635ff700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c06cc20 con 0x7fb95c0b1900 2023-11-28T11:09:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb9635ff700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c06be60 con 0x7fb95c15e000 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb96139b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb95c0b1900 0x7fb95c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb960b9a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb95c0b5600 0x7fb95c0cb330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48392/0 (socket says 172.21.15.83:48392) 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb960b9a700 1 -- 172.21.15.83:0/1816760142 learned_addr learned my addr 172.21.15.83:0/1816760142 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb953fff700 1 -- 172.21.15.83:0/1816760142 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131837818 0 0) 0x7fb95c06bc10 con 0x7fb95c0b5600 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb953fff700 1 -- 172.21.15.83:0/1816760142 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9480035b0 con 0x7fb95c0b5600 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb96139b700 1 -- 172.21.15.83:0/1816760142 >> v2:172.21.15.90:3300/0 conn(0x7fb95c15e000 msgr2=0x7fb95c15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb961b9c700 1 --2- 172.21.15.83:0/1816760142 >> v2:172.21.15.90:3300/0 conn(0x7fb95c15e000 0x7fb95c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb96139b700 1 --2- 172.21.15.83:0/1816760142 >> v2:172.21.15.90:3300/0 conn(0x7fb95c15e000 0x7fb95c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb96139b700 1 -- 172.21.15.83:0/1816760142 >> v1:172.21.15.83:6789/0 conn(0x7fb95c0b5600 legacy=0x7fb95c0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb96139b700 1 -- 172.21.15.83:0/1816760142 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c06c140 con 0x7fb95c0b1900 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.427+0000 7fb96139b700 1 --2- 172.21.15.83:0/1816760142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c0b50c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb95c06cfe0 tx=0x7fb958009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c7cf5ca8e04c05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb953fff700 1 -- 172.21.15.83:0/1816760142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958011830 con 0x7fb95c0b1900 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb953fff700 1 -- 172.21.15.83:0/1816760142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb958011990 con 0x7fb95c0b1900 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb953fff700 1 -- 172.21.15.83:0/1816760142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958011d20 con 0x7fb95c0b1900 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 -- 172.21.15.83:0/1816760142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 msgr2=0x7fb95c0b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 --2- 172.21.15.83:0/1816760142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c0b50c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb95c06cfe0 tx=0x7fb958009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 -- 172.21.15.83:0/1816760142 shutdown_connections 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 --2- 172.21.15.83:0/1816760142 >> v2:172.21.15.90:3300/0 conn(0x7fb95c15e000 0x7fb95c15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 --2- 172.21.15.83:0/1816760142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c0b50c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 -- 172.21.15.83:0/1816760142 >> 172.21.15.83:0/1816760142 conn(0x7fb95c0cc690 msgr2=0x7fb95c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 -- 172.21.15.83:0/1816760142 shutdown_connections 2023-11-28T11:09:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.428+0000 7fb9635ff700 1 -- 172.21.15.83:0/1816760142 wait complete. 2023-11-28T11:09:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.429+0000 7fb9635ff700 1 Processor -- start 2023-11-28T11:09:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.429+0000 7fb9635ff700 1 -- start start 2023-11-28T11:09:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb9635ff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c14e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb9635ff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb95c15e000 0x7fb95c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb9635ff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb95c06b040 0x7fb95c0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb9635ff700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c14e8b0 con 0x7fb95c0b1900 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb9635ff700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c06b4e0 con 0x7fb95c15e000 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb9635ff700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0c6ce0 con 0x7fb95c06b040 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb96139b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb96139b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c14e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50538/0 (socket says 172.21.15.83:50538) 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb96139b700 1 -- 172.21.15.83:0/2389987137 learned_addr learned my addr 172.21.15.83:0/2389987137 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb96139b700 1 -- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb95c06b040 msgr2=0x7fb95c0c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.430+0000 7fb96139b700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb95c06b040 0x7fb95c0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb960b9a700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb95c15e000 0x7fb95c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb96139b700 1 -- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb95c15e000 msgr2=0x7fb95c06ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb96139b700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb95c15e000 0x7fb95c06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb96139b700 1 -- 172.21.15.83:0/2389987137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580094a0 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb96139b700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c14e000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb958009b60 tx=0x7fb958004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9580036a0 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95801f070 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c06c700 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958003a40 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.431+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c06c420 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.432+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c15ab00 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.432+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb95800f050 con 0x7fb95c0b1900 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.433+0000 7fb951ffb700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9400682f0 0x7fb94006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.433+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb958021070 con 0x7fb95c0b1900 2023-11-28T11:09:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.433+0000 7fb960b9a700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9400682f0 0x7fb94006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.433+0000 7fb960b9a700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9400682f0 0x7fb94006a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb95c158ff0 tx=0x7fb94c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.437+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb958053db0 con 0x7fb95c0b1900 2023-11-28T11:09:41.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.698+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fb95c0742b0 con 0x7fb95c0b1900 2023-11-28T11:09:41.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.702+0000 7fb951ffb700 1 -- 172.21.15.83:0/2389987137 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb9580150a0 con 0x7fb95c0b1900 2023-11-28T11:09:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9400682f0 msgr2=0x7fb94006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9400682f0 0x7fb94006a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb95c158ff0 tx=0x7fb94c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 msgr2=0x7fb95c14e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c14e000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb958009b60 tx=0x7fb958004750 comp rx=0 tx=0).stop 2023-11-28T11:09:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 shutdown_connections 2023-11-28T11:09:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb95c06b040 0x7fb95c0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9400682f0 0x7fb94006a7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95c0b1900 0x7fb95c14e000 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 --2- 172.21.15.83:0/2389987137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb95c15e000 0x7fb95c06ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 >> 172.21.15.83:0/2389987137 conn(0x7fb95c0cc690 msgr2=0x7fb95c0cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 shutdown_connections 2023-11-28T11:09:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.704+0000 7fb9635ff700 1 -- 172.21.15.83:0/2389987137 wait complete. 2023-11-28T11:09:41.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T11:09:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.837+0000 7feb77c8e700 1 Processor -- start 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.837+0000 7feb77c8e700 1 -- start start 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb77c8e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb700cc020 0x7feb700cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb77c8e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb700cc930 con 0x7feb700c9bc0 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb77c8e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7015e4f0 con 0x7feb7015dff0 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb77c8e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb700cbc80 con 0x7feb700cc020 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb75a2a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb700cc020 0x7feb700cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb7622b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feb700c9bc0 0x7feb7006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48410/0 (socket says 172.21.15.83:48410) 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb7622b700 1 -- 172.21.15.83:0/333621724 learned_addr learned my addr 172.21.15.83:0/333621724 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb75a2a700 1 -- 172.21.15.83:0/333621724 >> v1:172.21.15.90:6789/0 conn(0x7feb7015dff0 legacy=0x7feb700cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb75a2a700 1 -- 172.21.15.83:0/333621724 >> v1:172.21.15.83:6789/0 conn(0x7feb700c9bc0 legacy=0x7feb7006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb74a28700 1 -- 172.21.15.83:0/333621724 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771722347 0 0) 0x7feb700cc930 con 0x7feb700c9bc0 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.838+0000 7feb75a2a700 1 -- 172.21.15.83:0/333621724 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7015e7d0 con 0x7feb700cc020 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb75a2a700 1 --2- 172.21.15.83:0/333621724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700cc020 0x7feb700cc3f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feb60009840 tx=0x7feb60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb57e362a0edf08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb74a28700 1 -- 172.21.15.83:0/333621724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60011830 con 0x7feb700cc020 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb74a28700 1 -- 172.21.15.83:0/333621724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60011990 con 0x7feb700cc020 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb74a28700 1 -- 172.21.15.83:0/333621724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60011d20 con 0x7feb700cc020 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 -- 172.21.15.83:0/333621724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700cc020 msgr2=0x7feb700cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 --2- 172.21.15.83:0/333621724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700cc020 0x7feb700cc3f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feb60009840 tx=0x7feb60009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 -- 172.21.15.83:0/333621724 shutdown_connections 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 --2- 172.21.15.83:0/333621724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700cc020 0x7feb700cc3f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 -- 172.21.15.83:0/333621724 >> 172.21.15.83:0/333621724 conn(0x7feb700b1090 msgr2=0x7feb700b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 -- 172.21.15.83:0/333621724 shutdown_connections 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.839+0000 7feb77c8e700 1 -- 172.21.15.83:0/333621724 wait complete. 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 Processor -- start 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 -- start start 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 0x7feb701ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb700cc020 0x7feb7015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb7015dff0 0x7feb7015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb701e6920 con 0x7feb700c9bc0 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb701e5da0 con 0x7feb700cc020 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb77c8e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb701e6080 con 0x7feb7015dff0 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb75a2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 0x7feb701ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb75a2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 0x7feb701ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50552/0 (socket says 172.21.15.83:50552) 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.841+0000 7feb75a2a700 1 -- 172.21.15.83:0/3646919506 learned_addr learned my addr 172.21.15.83:0/3646919506 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb7622b700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb7015dff0 0x7feb7015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75229700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb700cc020 0x7feb7015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75a2a700 1 -- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb7015dff0 msgr2=0x7feb7015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75a2a700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb7015dff0 0x7feb7015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75a2a700 1 -- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb700cc020 msgr2=0x7feb7015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75a2a700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb700cc020 0x7feb7015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75a2a700 1 -- 172.21.15.83:0/3646919506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600094a0 con 0x7feb700c9bc0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb7622b700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb7015dff0 0x7feb7015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75229700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb700cc020 0x7feb7015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb75a2a700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 0x7feb701ef400 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb600036a0 con 0x7feb700c9bc0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6001f070 con 0x7feb700c9bc0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7015ed90 con 0x7feb700c9bc0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60003a00 con 0x7feb700c9bc0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.842+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7015eab0 con 0x7feb700c9bc0 2023-11-28T11:09:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.843+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7015aa00 con 0x7feb700c9bc0 2023-11-28T11:09:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.843+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb6000f050 con 0x7feb700c9bc0 2023-11-28T11:09:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.844+0000 7feb667fc700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb540682f0 0x7feb5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.844+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7feb60021070 con 0x7feb700c9bc0 2023-11-28T11:09:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.844+0000 7feb75229700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb540682f0 0x7feb5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.844+0000 7feb75229700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb540682f0 0x7feb5406a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feb701456e0 tx=0x7feb6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:41.848+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb60053f50 con 0x7feb700c9bc0 2023-11-28T11:09:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.104+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7feb700c3660 con 0x7feb700c9bc0 2023-11-28T11:09:42.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.108+0000 7feb667fc700 1 -- 172.21.15.83:0/3646919506 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7feb700c3660 con 0x7feb700c9bc0 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb540682f0 msgr2=0x7feb5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb540682f0 0x7feb5406a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feb701456e0 tx=0x7feb6c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 msgr2=0x7feb701ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 0x7feb701ef400 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb60004750 comp rx=0 tx=0).stop 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 shutdown_connections 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb7015dff0 0x7feb7015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb540682f0 0x7feb5406a7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700c9bc0 0x7feb701ef400 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 --2- 172.21.15.83:0/3646919506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb700cc020 0x7feb7015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 >> 172.21.15.83:0/3646919506 conn(0x7feb700b1090 msgr2=0x7feb70154890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 shutdown_connections 2023-11-28T11:09:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.110+0000 7feb77c8e700 1 -- 172.21.15.83:0/3646919506 wait complete. 2023-11-28T11:09:42.123 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T11:09:42.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.242+0000 7ff830e87700 1 Processor -- start 2023-11-28T11:09:42.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.242+0000 7ff830e87700 1 -- start start 2023-11-28T11:09:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.243+0000 7ff830e87700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff82c0aee50 0x7ff82c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.243+0000 7ff830e87700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff82c0af220 0x7ff82c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.243+0000 7ff830e87700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c0d0970 con 0x7ff82c15d8b0 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.243+0000 7ff830e87700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c15dcf0 con 0x7ff82c0aee50 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.243+0000 7ff830e87700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c15dee0 con 0x7ff82c0af220 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff82c0aee50 0x7ff82c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff82c0aee50 0x7ff82c0d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50560/0 (socket says 172.21.15.83:50560) 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 -- 172.21.15.83:0/3607323914 learned_addr learned my addr 172.21.15.83:0/3607323914 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 -- 172.21.15.83:0/3607323914 >> v2:172.21.15.90:3300/0 conn(0x7ff82c0af220 msgr2=0x7ff82c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff829d9b700 1 --2- 172.21.15.83:0/3607323914 >> v2:172.21.15.90:3300/0 conn(0x7ff82c0af220 0x7ff82c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 --2- 172.21.15.83:0/3607323914 >> v2:172.21.15.90:3300/0 conn(0x7ff82c0af220 0x7ff82c15d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 -- 172.21.15.83:0/3607323914 >> v1:172.21.15.90:6789/0 conn(0x7ff82c15d8b0 legacy=0x7ff82c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 -- 172.21.15.83:0/3607323914 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0cf950 con 0x7ff82c0aee50 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82a59c700 1 --2- 172.21.15.83:0/3607323914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c0aee50 0x7ff82c0d0430 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff82c070b50 tx=0x7ff82000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df1d2a466b1716b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82959a700 1 -- 172.21.15.83:0/3607323914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82000dcf0 con 0x7ff82c0aee50 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82959a700 1 -- 172.21.15.83:0/3607323914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82000de50 con 0x7ff82c0aee50 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.244+0000 7ff82959a700 1 -- 172.21.15.83:0/3607323914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820004250 con 0x7ff82c0aee50 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 -- 172.21.15.83:0/3607323914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c0aee50 msgr2=0x7ff82c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 --2- 172.21.15.83:0/3607323914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c0aee50 0x7ff82c0d0430 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff82c070b50 tx=0x7ff82000be70 comp rx=0 tx=0).stop 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 -- 172.21.15.83:0/3607323914 shutdown_connections 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 --2- 172.21.15.83:0/3607323914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c0aee50 0x7ff82c0d0430 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 --2- 172.21.15.83:0/3607323914 >> v2:172.21.15.90:3300/0 conn(0x7ff82c0af220 0x7ff82c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 -- 172.21.15.83:0/3607323914 >> 172.21.15.83:0/3607323914 conn(0x7ff82c0b2b30 msgr2=0x7ff82c0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 -- 172.21.15.83:0/3607323914 shutdown_connections 2023-11-28T11:09:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.245+0000 7ff830e87700 1 -- 172.21.15.83:0/3607323914 wait complete. 2023-11-28T11:09:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.246+0000 7ff830e87700 1 Processor -- start 2023-11-28T11:09:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.246+0000 7ff830e87700 1 -- start start 2023-11-28T11:09:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff830e87700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff82c0aee50 0x7ff82c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff830e87700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff82c0af220 0x7ff82c1f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff830e87700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 0x7ff82c1f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff830e87700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c15cc70 con 0x7ff82c15d8b0 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff830e87700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c15c6b0 con 0x7ff82c0af220 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff830e87700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c15c990 con 0x7ff82c0aee50 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 0x7ff82c1f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 0x7ff82c1f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50568/0 (socket says 172.21.15.83:50568) 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 -- 172.21.15.83:0/2073907637 learned_addr learned my addr 172.21.15.83:0/2073907637 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82a59c700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff82c0aee50 0x7ff82c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff829d9b700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff82c0af220 0x7ff82c1f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 -- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff82c0aee50 msgr2=0x7ff82c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff82c0aee50 0x7ff82c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 -- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff82c0af220 msgr2=0x7ff82c1f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff82c0af220 0x7ff82c1f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.247+0000 7ff82ad9d700 1 -- 172.21.15.83:0/2073907637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82000b7f0 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff829d9b700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff82c0af220 0x7ff82c1f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff82ad9d700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 0x7ff82c1f2390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff82400cc10 tx=0x7ff82400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82400ab10 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82400ac70 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0cff10 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824012420 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.248+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c0cfc30 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.249+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c0b1910 con 0x7ff82c15d8b0 2023-11-28T11:09:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.249+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff824010050 con 0x7ff82c15d8b0 2023-11-28T11:09:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.250+0000 7ff81affd700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff80c0682f0 0x7ff80c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.250+0000 7ff82a59c700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff80c0682f0 0x7ff80c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.250+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff824085dc0 con 0x7ff82c15d8b0 2023-11-28T11:09:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.250+0000 7ff82a59c700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff80c0682f0 0x7ff80c06a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b340 tx=0x7ff82001d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.254+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff824052740 con 0x7ff82c15d8b0 2023-11-28T11:09:42.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.508+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7ff82c0742b0 con 0x7ff82c15d8b0 2023-11-28T11:09:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.511+0000 7ff81affd700 1 -- 172.21.15.83:0/2073907637 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff824055d60 con 0x7ff82c15d8b0 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff80c0682f0 msgr2=0x7ff80c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff80c0682f0 0x7ff80c06a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b340 tx=0x7ff82001d040 comp rx=0 tx=0).stop 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 msgr2=0x7ff82c1f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 0x7ff82c1f2390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff82400cc10 tx=0x7ff82400cf20 comp rx=0 tx=0).stop 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 shutdown_connections 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff80c0682f0 0x7ff80c06a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff82c15d8b0 0x7ff82c1f2390 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff82c0aee50 0x7ff82c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 --2- 172.21.15.83:0/2073907637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff82c0af220 0x7ff82c1f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 >> 172.21.15.83:0/2073907637 conn(0x7ff82c0b2b30 msgr2=0x7ff82c06e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 shutdown_connections 2023-11-28T11:09:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.513+0000 7ff830e87700 1 -- 172.21.15.83:0/2073907637 wait complete. 2023-11-28T11:09:42.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T11:09:42.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.646+0000 7f6ab3074700 1 Processor -- start 2023-11-28T11:09:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.646+0000 7f6ab3074700 1 -- start start 2023-11-28T11:09:42.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab3074700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac15e5c0 0x7f6aac15e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab3074700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6aac0d11f0 0x7f6aac0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab3074700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac151a30 con 0x7f6aac15b3b0 2023-11-28T11:09:42.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab3074700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0cb460 con 0x7f6aac0d11f0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab3074700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0cb740 con 0x7f6aac15e5c0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6aabfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6aac15b3b0 0x7f6aac0d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48432/0 (socket says 172.21.15.83:48432) 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6aabfff700 1 -- 172.21.15.83:0/1734728346 learned_addr learned my addr 172.21.15.83:0/1734728346 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1734728346 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac15e5c0 0x7f6aac15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab1611700 1 --2- 172.21.15.83:0/1734728346 >> v2:172.21.15.90:3300/0 conn(0x7f6aac0d11f0 0x7f6aac0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6aab7fe700 1 -- 172.21.15.83:0/1734728346 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122147768 0 0) 0x7f6aac151a30 con 0x7f6aac15b3b0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6aab7fe700 1 -- 172.21.15.83:0/1734728346 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a9c0035b0 con 0x7f6aac15b3b0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab0e10700 1 -- 172.21.15.83:0/1734728346 >> v2:172.21.15.90:3300/0 conn(0x7f6aac0d11f0 msgr2=0x7f6aac0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1734728346 >> v2:172.21.15.90:3300/0 conn(0x7f6aac0d11f0 0x7f6aac0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab0e10700 1 -- 172.21.15.83:0/1734728346 >> v1:172.21.15.83:6789/0 conn(0x7f6aac15b3b0 legacy=0x7f6aac0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab0e10700 1 -- 172.21.15.83:0/1734728346 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac15abb0 con 0x7f6aac15e5c0 2023-11-28T11:09:42.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.647+0000 7f6ab1611700 1 --2- 172.21.15.83:0/1734728346 >> v2:172.21.15.90:3300/0 conn(0x7f6aac0d11f0 0x7f6aac0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1734728346 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac15e5c0 0x7f6aac15e990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009840 tx=0x7f6aa0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520b100b6069e094 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6aab7fe700 1 -- 172.21.15.83:0/1734728346 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0011830 con 0x7f6aac15e5c0 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6aab7fe700 1 -- 172.21.15.83:0/1734728346 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0011990 con 0x7f6aac15e5c0 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6aab7fe700 1 -- 172.21.15.83:0/1734728346 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0011d20 con 0x7f6aac15e5c0 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 -- 172.21.15.83:0/1734728346 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac15e5c0 msgr2=0x7f6aac15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1734728346 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac15e5c0 0x7f6aac15e990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009840 tx=0x7f6aa0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 -- 172.21.15.83:0/1734728346 shutdown_connections 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1734728346 >> v2:172.21.15.90:3300/0 conn(0x7f6aac0d11f0 0x7f6aac0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1734728346 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac15e5c0 0x7f6aac15e990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 -- 172.21.15.83:0/1734728346 >> 172.21.15.83:0/1734728346 conn(0x7f6aac06ea50 msgr2=0x7f6aac070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 -- 172.21.15.83:0/1734728346 shutdown_connections 2023-11-28T11:09:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.648+0000 7f6ab3074700 1 -- 172.21.15.83:0/1734728346 wait complete. 2023-11-28T11:09:42.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.649+0000 7f6ab3074700 1 Processor -- start 2023-11-28T11:09:42.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.649+0000 7f6ab3074700 1 -- start start 2023-11-28T11:09:42.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6ab3074700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6aac15b3b0 0x7f6aac15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6ab3074700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 0x7f6aac1e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6ab3074700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac14e0c0 0x7f6aac0c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6ab3074700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0ca070 con 0x7f6aac15e5c0 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6ab3074700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1e6c30 con 0x7f6aac15b3b0 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6ab3074700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0cac80 con 0x7f6aac14e0c0 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6aabfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 0x7f6aac1e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6aabfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 0x7f6aac1e3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50578/0 (socket says 172.21.15.83:50578) 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.650+0000 7f6aabfff700 1 -- 172.21.15.83:0/1500150522 learned_addr learned my addr 172.21.15.83:0/1500150522 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6ab1611700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac14e0c0 0x7f6aac0c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6aac15b3b0 0x7f6aac15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aabfff700 1 -- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac14e0c0 msgr2=0x7f6aac0c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aabfff700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac14e0c0 0x7f6aac0c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aabfff700 1 -- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6aac15b3b0 msgr2=0x7f6aac15c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aabfff700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6aac15b3b0 0x7f6aac15c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aabfff700 1 -- 172.21.15.83:0/1500150522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4009a70 con 0x7f6aac15e5c0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6aac15b3b0 0x7f6aac15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aabfff700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 0x7f6aac1e3d70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c730 tx=0x7f6a9800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005d70 con 0x7f6aac15e5c0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a980070d0 con 0x7f6aac15e5c0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa00094a0 con 0x7f6aac15e5c0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98007440 con 0x7f6aac15e5c0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.651+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac15eed0 con 0x7f6aac15e5c0 2023-11-28T11:09:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.652+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac15ae90 con 0x7f6aac15e5c0 2023-11-28T11:09:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.652+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a98019030 con 0x7f6aac15e5c0 2023-11-28T11:09:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.653+0000 7f6aa97fa700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a900682f0 0x7f6a9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.653+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6a98084d40 con 0x7f6aac15e5c0 2023-11-28T11:09:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.653+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a900682f0 0x7f6a9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.653+0000 7f6ab0e10700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a900682f0 0x7f6a9006a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009b90 tx=0x7f6aa0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.657+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a9804b5e0 con 0x7f6aac15e5c0 2023-11-28T11:09:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.915+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6aac0cba20 con 0x7f6aac15e5c0 2023-11-28T11:09:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.918+0000 7f6aa97fa700 1 -- 172.21.15.83:0/1500150522 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6a980170a0 con 0x7f6aac15e5c0 2023-11-28T11:09:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.920+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a900682f0 msgr2=0x7f6a9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.920+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a900682f0 0x7f6a9006a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009b90 tx=0x7f6aa0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.920+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 msgr2=0x7f6aac1e3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.920+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 0x7f6aac1e3d70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c730 tx=0x7f6a9800caf0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 shutdown_connections 2023-11-28T11:09:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6aac14e0c0 0x7f6aac0c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6aac15b3b0 0x7f6aac15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6a900682f0 0x7f6a9006a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 --2- 172.21.15.83:0/1500150522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aac15e5c0 0x7f6aac1e3d70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 >> 172.21.15.83:0/1500150522 conn(0x7f6aac0d11f0 msgr2=0x7f6aac070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 shutdown_connections 2023-11-28T11:09:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:42.921+0000 7f6ab3074700 1 -- 172.21.15.83:0/1500150522 wait complete. 2023-11-28T11:09:42.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T11:09:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.053+0000 7f5dcc6ec700 1 Processor -- start 2023-11-28T11:09:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.053+0000 7f5dcc6ec700 1 -- start start 2023-11-28T11:09:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.054+0000 7f5dcc6ec700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc41597d0 0x7f5dc406c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.054+0000 7f5dcc6ec700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dc40cbdf0 0x7f5dc406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.054+0000 7f5dcc6ec700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5dc406cd80 0x7f5dc40cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.054+0000 7f5dcc6ec700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40cb960 con 0x7f5dc40cbdf0 2023-11-28T11:09:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.054+0000 7f5dcc6ec700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40cad50 con 0x7f5dc406cd80 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.054+0000 7f5dcc6ec700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40cb030 con 0x7f5dc41597d0 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dc40cbdf0 0x7f5dc406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dc40cbdf0 0x7f5dc406c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50586/0 (socket says 172.21.15.83:50586) 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 -- 172.21.15.83:0/1722426073 learned_addr learned my addr 172.21.15.83:0/1722426073 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dca488700 1 --2- 172.21.15.83:0/1722426073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc41597d0 0x7f5dc406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dcac89700 1 --2- 172.21.15.83:0/1722426073 >> v2:172.21.15.90:3300/0 conn(0x7f5dc406cd80 0x7f5dc40cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 -- 172.21.15.83:0/1722426073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc41597d0 msgr2=0x7f5dc406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 --2- 172.21.15.83:0/1722426073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc41597d0 0x7f5dc406c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 -- 172.21.15.83:0/1722426073 >> v2:172.21.15.90:3300/0 conn(0x7f5dc406cd80 msgr2=0x7f5dc40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 --2- 172.21.15.83:0/1722426073 >> v2:172.21.15.90:3300/0 conn(0x7f5dc406cd80 0x7f5dc40cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 -- 172.21.15.83:0/1722426073 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40cb310 con 0x7f5dc40cbdf0 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9c87700 1 --2- 172.21.15.83:0/1722426073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc40cbdf0 0x7f5dc406c840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4071c40 tx=0x7f5dc000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f0f067baac732f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9486700 1 -- 172.21.15.83:0/1722426073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000dc70 con 0x7f5dc40cbdf0 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9486700 1 -- 172.21.15.83:0/1722426073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc000ddd0 con 0x7f5dc40cbdf0 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.055+0000 7f5dc9486700 1 -- 172.21.15.83:0/1722426073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0010690 con 0x7f5dc40cbdf0 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/1722426073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc40cbdf0 msgr2=0x7f5dc406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/1722426073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc40cbdf0 0x7f5dc406c840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4071c40 tx=0x7f5dc000be70 comp rx=0 tx=0).stop 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/1722426073 shutdown_connections 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/1722426073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc41597d0 0x7f5dc406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/1722426073 >> v2:172.21.15.90:3300/0 conn(0x7f5dc406cd80 0x7f5dc40cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/1722426073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc40cbdf0 0x7f5dc406c840 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/1722426073 >> 172.21.15.83:0/1722426073 conn(0x7f5dc40b2b10 msgr2=0x7f5dc40b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/1722426073 shutdown_connections 2023-11-28T11:09:43.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.056+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/1722426073 wait complete. 2023-11-28T11:09:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.057+0000 7f5dcc6ec700 1 Processor -- start 2023-11-28T11:09:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.057+0000 7f5dcc6ec700 1 -- start start 2023-11-28T11:09:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcc6ec700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 0x7f5dc4155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcc6ec700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc40cbdf0 0x7f5dc41576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcc6ec700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dc41597d0 0x7f5dc4157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcc6ec700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40ce870 con 0x7f5dc406cd80 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcc6ec700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41e47d0 con 0x7f5dc41597d0 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcc6ec700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c42d0 con 0x7f5dc40cbdf0 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 0x7f5dc4155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 0x7f5dc4155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50590/0 (socket says 172.21.15.83:50590) 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 -- 172.21.15.83:0/2203606157 learned_addr learned my addr 172.21.15.83:0/2203606157 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dc9c87700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc40cbdf0 0x7f5dc41576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dcac89700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dc41597d0 0x7f5dc4157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 -- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc40cbdf0 msgr2=0x7f5dc41576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc40cbdf0 0x7f5dc41576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 -- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dc41597d0 msgr2=0x7f5dc4157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dc41597d0 0x7f5dc4157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 -- 172.21.15.83:0/2203606157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc000b7f0 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dca488700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 0x7f5dc4155240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e870 tx=0x7f5db400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4009590 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.058+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400ee70 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.059+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc40aea90 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.059+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400ca50 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.059+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40cb5f0 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.060+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc40aec20 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.060+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5db4019030 con 0x7f5dc406cd80 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.060+0000 7f5dbaffd700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5da80682f0 0x7f5da806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.060+0000 7f5dc9c87700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5da80682f0 0x7f5da806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.060+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5db4085960 con 0x7f5dc406cd80 2023-11-28T11:09:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.061+0000 7f5dc9c87700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5da80682f0 0x7f5da806a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cc200 tx=0x7f5dc001b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.065+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5db4052370 con 0x7f5dc406cd80 2023-11-28T11:09:43.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.322+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f5dc40cd660 con 0x7f5dc406cd80 2023-11-28T11:09:43.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.325+0000 7f5dbaffd700 1 -- 172.21.15.83:0/2203606157 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5db40170a0 con 0x7f5dc406cd80 2023-11-28T11:09:43.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.326+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5da80682f0 msgr2=0x7f5da806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.326+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5da80682f0 0x7f5da806a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cc200 tx=0x7f5dc001b040 comp rx=0 tx=0).stop 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 msgr2=0x7f5dc4155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 0x7f5dc4155240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e870 tx=0x7f5db400ec30 comp rx=0 tx=0).stop 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 shutdown_connections 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dc41597d0 0x7f5dc4157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5da80682f0 0x7f5da806a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc406cd80 0x7f5dc4155240 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 --2- 172.21.15.83:0/2203606157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dc40cbdf0 0x7f5dc41576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 >> 172.21.15.83:0/2203606157 conn(0x7f5dc40b2b10 msgr2=0x7f5dc406ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 shutdown_connections 2023-11-28T11:09:43.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.327+0000 7f5dcc6ec700 1 -- 172.21.15.83:0/2203606157 wait complete. 2023-11-28T11:09:43.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.458+0000 7f1990642700 1 Processor -- start 2023-11-28T11:09:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.458+0000 7f1990642700 1 -- start start 2023-11-28T11:09:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.459+0000 7f1990642700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f198815d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.459+0000 7f1990642700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f19880cf1c0 0x7f198815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.459+0000 7f1990642700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1988161220 con 0x7f198815e060 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.459+0000 7f1990642700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198815de60 con 0x7f19880cf1c0 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.459+0000 7f1990642700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198815cce0 con 0x7f19880cdd20 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f19880cf1c0 0x7f198815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f19880cf1c0 0x7f198815f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50602/0 (socket says 172.21.15.83:50602) 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 -- 172.21.15.83:0/4164396839 learned_addr learned my addr 172.21.15.83:0/4164396839 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198e3de700 1 --2- 172.21.15.83:0/4164396839 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f198815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 -- 172.21.15.83:0/4164396839 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 msgr2=0x7f198815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 --2- 172.21.15.83:0/4164396839 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f198815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 -- 172.21.15.83:0/4164396839 >> v1:172.21.15.90:6789/0 conn(0x7f198815e060 legacy=0x7f19880ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 -- 172.21.15.83:0/4164396839 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988160610 con 0x7f19880cf1c0 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198ebdf700 1 --2- 172.21.15.83:0/4164396839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f198815f4c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1980009e50 tx=0x7f198000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02130c3458f9967 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198d3dc700 1 -- 172.21.15.83:0/4164396839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000e070 con 0x7f19880cf1c0 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198d3dc700 1 -- 172.21.15.83:0/4164396839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198000b9c0 con 0x7f19880cf1c0 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.460+0000 7f198d3dc700 1 -- 172.21.15.83:0/4164396839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000bd70 con 0x7f19880cf1c0 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 -- 172.21.15.83:0/4164396839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 msgr2=0x7f198815f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 --2- 172.21.15.83:0/4164396839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f198815f4c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1980009e50 tx=0x7f198000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 -- 172.21.15.83:0/4164396839 shutdown_connections 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 --2- 172.21.15.83:0/4164396839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f198815f4c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 --2- 172.21.15.83:0/4164396839 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f198815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 -- 172.21.15.83:0/4164396839 >> 172.21.15.83:0/4164396839 conn(0x7f198806de20 msgr2=0x7f198806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 -- 172.21.15.83:0/4164396839 shutdown_connections 2023-11-28T11:09:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.461+0000 7f1990642700 1 -- 172.21.15.83:0/4164396839 wait complete. 2023-11-28T11:09:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.462+0000 7f1990642700 1 Processor -- start 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.462+0000 7f1990642700 1 -- start start 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f19880ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f19880afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f198815e060 0x7f198815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880af610 con 0x7f19880cf1c0 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880b0110 con 0x7f198815e060 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880ae990 con 0x7f19880cdd20 2023-11-28T11:09:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f19880afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f19880afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50606/0 (socket says 172.21.15.83:50606) 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 -- 172.21.15.83:0/2128877115 learned_addr learned my addr 172.21.15.83:0/2128877115 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198ebdf700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f198815e060 0x7f198815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198e3de700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f19880ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 -- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 msgr2=0x7f19880ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f19880ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 -- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f198815e060 msgr2=0x7f198815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f198815e060 0x7f198815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 -- 172.21.15.83:0/2128877115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009b30 con 0x7f19880cf1c0 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f198dbdd700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f19880afbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f198400c4e0 tx=0x7f198400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840052d0 con 0x7f19880cf1c0 2023-11-28T11:09:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984005430 con 0x7f19880cf1c0 2023-11-28T11:09:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988160bd0 con 0x7f19880cf1c0 2023-11-28T11:09:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.463+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840038c0 con 0x7f19880cf1c0 2023-11-28T11:09:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.464+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881608f0 con 0x7f19880cf1c0 2023-11-28T11:09:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.464+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1988160eb0 con 0x7f19880cf1c0 2023-11-28T11:09:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.466+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1984019030 con 0x7f19880cf1c0 2023-11-28T11:09:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.466+0000 7f197effd700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f196c0682f0 0x7f196c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.466+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1984085100 con 0x7f19880cf1c0 2023-11-28T11:09:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.466+0000 7f198e3de700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f196c0682f0 0x7f196c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.466+0000 7f198e3de700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f196c0682f0 0x7f196c06a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1978006b20 tx=0x7f1978008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.469+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1984051c10 con 0x7f19880cf1c0 2023-11-28T11:09:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.655+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1988074320 con 0x7f19880cf1c0 2023-11-28T11:09:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.656+0000 7f197effd700 1 -- 172.21.15.83:0/2128877115 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+14366 (secure 0 0 0) 0x7f1984055230 con 0x7f19880cf1c0 2023-11-28T11:09:43.658 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:43.658 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:40.426252+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f196c0682f0 msgr2=0x7f196c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f196c0682f0 0x7f196c06a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1978006b20 tx=0x7f1978008040 comp rx=0 tx=0).stop 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 msgr2=0x7f19880afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f19880afbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f198400c4e0 tx=0x7f198400c7f0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 shutdown_connections 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f198815e060 0x7f198815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f19880cdd20 0x7f19880ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f196c0682f0 0x7f196c06a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 --2- 172.21.15.83:0/2128877115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19880cf1c0 0x7f19880afbd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 >> 172.21.15.83:0/2128877115 conn(0x7f198806de20 msgr2=0x7f19880ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 shutdown_connections 2023-11-28T11:09:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.658+0000 7f1990642700 1 -- 172.21.15.83:0/2128877115 wait complete. 2023-11-28T11:09:43.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:43.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.795+0000 7f88e0b74700 1 Processor -- start 2023-11-28T11:09:43.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.795+0000 7f88e0b74700 1 -- start start 2023-11-28T11:09:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88e0b74700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88dc15cad0 0x7f88dc0d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88e0b74700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88e0b74700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88dc0cc770 con 0x7f88dc0af1d0 2023-11-28T11:09:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88e0b74700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88dc0cd3f0 con 0x7f88dc15cad0 2023-11-28T11:09:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88e0b74700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0cc570 con 0x7f88dc0b1d70 2023-11-28T11:09:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88da59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88dc15cad0 0x7f88dc0d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88da59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88dc15cad0 0x7f88dc0d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50612/0 (socket says 172.21.15.83:50612) 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88d9d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f88dc0af1d0 0x7f88dc0b1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48472/0 (socket says 172.21.15.83:48472) 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88da59c700 1 -- 172.21.15.83:0/2330335688 learned_addr learned my addr 172.21.15.83:0/2330335688 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.796+0000 7f88dad9d700 1 --2- 172.21.15.83:0/2330335688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88da59c700 1 -- 172.21.15.83:0/2330335688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 msgr2=0x7f88dc0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88da59c700 1 --2- 172.21.15.83:0/2330335688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88da59c700 1 -- 172.21.15.83:0/2330335688 >> v1:172.21.15.83:6789/0 conn(0x7f88dc0af1d0 legacy=0x7f88dc0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88da59c700 1 -- 172.21.15.83:0/2330335688 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc0cca50 con 0x7f88dc15cad0 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88d959a700 1 -- 172.21.15.83:0/2330335688 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255498016 0 0) 0x7f88dc0cc770 con 0x7f88dc0af1d0 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88da59c700 1 --2- 172.21.15.83:0/2330335688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0d02b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f88d00098f0 tx=0x7f88d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9176c87dd52b88b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88d959a700 1 -- 172.21.15.83:0/2330335688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d000b8a0 con 0x7f88dc15cad0 2023-11-28T11:09:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88d959a700 1 -- 172.21.15.83:0/2330335688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d000ba00 con 0x7f88dc15cad0 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88d959a700 1 -- 172.21.15.83:0/2330335688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d000bd90 con 0x7f88dc15cad0 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88e0b74700 1 -- 172.21.15.83:0/2330335688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 msgr2=0x7f88dc0d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88e0b74700 1 --2- 172.21.15.83:0/2330335688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0d02b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f88d00098f0 tx=0x7f88d0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88e0b74700 1 -- 172.21.15.83:0/2330335688 shutdown_connections 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88e0b74700 1 --2- 172.21.15.83:0/2330335688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88e0b74700 1 --2- 172.21.15.83:0/2330335688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0d02b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.797+0000 7f88e0b74700 1 -- 172.21.15.83:0/2330335688 >> 172.21.15.83:0/2330335688 conn(0x7f88dc0b2b30 msgr2=0x7f88dc0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.798+0000 7f88e0b74700 1 -- 172.21.15.83:0/2330335688 shutdown_connections 2023-11-28T11:09:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.798+0000 7f88e0b74700 1 -- 172.21.15.83:0/2330335688 wait complete. 2023-11-28T11:09:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.799+0000 7f88e0b74700 1 Processor -- start 2023-11-28T11:09:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.799+0000 7f88e0b74700 1 -- start start 2023-11-28T11:09:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88dc0af1d0 0x7f88dc15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc15afd0 con 0x7f88dc15cad0 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0c7d30 con 0x7f88dc0af1d0 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0c7770 con 0x7f88dc0b1d70 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50622/0 (socket says 172.21.15.83:50622) 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 -- 172.21.15.83:0/3812534918 learned_addr learned my addr 172.21.15.83:0/3812534918 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88d9d9b700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88da59c700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88dc0af1d0 0x7f88dc15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 -- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 msgr2=0x7f88dc0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 -- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88dc0af1d0 msgr2=0x7f88dc15aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88dc0af1d0 0x7f88dc15aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 -- 172.21.15.83:0/3812534918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009550 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88dad9d700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0c8970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f88d4007e60 tx=0x7f88d400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400ede0 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4014420 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d40147d0 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88dc0cd010 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.800+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88dc0ccd30 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.801+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88dc15cea0 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.802+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88d4012050 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.802+0000 7f88caffd700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88bc0682f0 0x7f88bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.802+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f88d4086020 con 0x7f88dc15cad0 2023-11-28T11:09:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.802+0000 7f88da59c700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88bc0682f0 0x7f88bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.802+0000 7f88da59c700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88bc0682f0 0x7f88bc06a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f88d0014040 tx=0x7f88d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.806+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88d4052a90 con 0x7f88dc15cad0 2023-11-28T11:09:43.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.986+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f88dc0742b0 con 0x7f88dc15cad0 2023-11-28T11:09:43.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.986+0000 7f88caffd700 1 -- 172.21.15.83:0/3812534918 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f88d40560b0 con 0x7f88dc15cad0 2023-11-28T11:09:43.988 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88bc0682f0 msgr2=0x7f88bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88bc0682f0 0x7f88bc06a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f88d0014040 tx=0x7f88d0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 msgr2=0x7f88dc0c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0c8970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f88d4007e60 tx=0x7f88d400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 shutdown_connections 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88bc0682f0 0x7f88bc06a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc15cad0 0x7f88dc0c8970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88dc0af1d0 0x7f88dc15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 --2- 172.21.15.83:0/3812534918 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88dc0b1d70 0x7f88dc0c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 >> 172.21.15.83:0/3812534918 conn(0x7f88dc0b2b30 msgr2=0x7f88dc06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 shutdown_connections 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:43.988+0000 7f88e0b74700 1 -- 172.21.15.83:0/3812534918 wait complete. 2023-11-28T11:09:44.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.123+0000 7fbf4920e700 1 Processor -- start 2023-11-28T11:09:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.123+0000 7fbf4920e700 1 -- start start 2023-11-28T11:09:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4920e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbf440aee60 0x7fbf440b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4920e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4406d4a0 con 0x7fbf4406e370 2023-11-28T11:09:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4920e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf440b5ae0 con 0x7fbf4415c890 2023-11-28T11:09:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4920e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4406c400 con 0x7fbf440aee60 2023-11-28T11:09:44.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbf4406e370 0x7fbf4406b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48488/0 (socket says 172.21.15.83:48488) 2023-11-28T11:09:44.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4259c700 1 -- 172.21.15.83:0/120932124 learned_addr learned my addr 172.21.15.83:0/120932124 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:44.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4359e700 1 --2- 172.21.15.83:0/120932124 >> v2:172.21.15.90:3300/0 conn(0x7fbf440aee60 0x7fbf440b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645169764 0 0) 0x7fbf4406d4a0 con 0x7fbf4406e370 2023-11-28T11:09:44.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf2c0035b0 con 0x7fbf4406e370 2023-11-28T11:09:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 861018683 0 0) 0x7fbf2c0035b0 con 0x7fbf4406e370 2023-11-28T11:09:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 >> v2:172.21.15.90:3300/0 conn(0x7fbf440aee60 msgr2=0x7fbf440b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 --2- 172.21.15.83:0/120932124 >> v2:172.21.15.90:3300/0 conn(0x7fbf440aee60 0x7fbf440b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 >> v1:172.21.15.90:6789/0 conn(0x7fbf4415c890 legacy=0x7fbf4406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440b5240 con 0x7fbf4406e370 2023-11-28T11:09:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fbf38001670 con 0x7fbf4406e370 2023-11-28T11:09:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 >> v1:172.21.15.83:6789/0 conn(0x7fbf4406e370 legacy=0x7fbf4406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf2c003a00 0x7fbf2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2c004850 0x7fbf2c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c005200 0x7fbf2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4406c400 con 0x7fbf2c004850 2023-11-28T11:09:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c00c110 con 0x7fbf2c003a00 2023-11-28T11:09:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c00c2d0 con 0x7fbf2c005200 2023-11-28T11:09:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf4259c700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2c004850 0x7fbf2c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.124+0000 7fbf42d9d700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf2c003a00 0x7fbf2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4359e700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c005200 0x7fbf2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4259c700 1 -- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c005200 msgr2=0x7fbf2c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4259c700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c005200 0x7fbf2c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4259c700 1 -- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf2c003a00 msgr2=0x7fbf2c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4259c700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf2c003a00 0x7fbf2c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4259c700 1 -- 172.21.15.83:0/120932124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0035b0 con 0x7fbf2c004850 2023-11-28T11:09:44.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4359e700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c005200 0x7fbf2c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4359e700 1 -- 172.21.15.83:0/120932124 reap_dead start 2023-11-28T11:09:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4259c700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2c004850 0x7fbf2c004cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800c210 tx=0x7fbf2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb66ddf04f5834a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38007c10 con 0x7fbf2c004850 2023-11-28T11:09:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800cdb0 con 0x7fbf2c004850 2023-11-28T11:09:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf41d9b700 1 -- 172.21.15.83:0/120932124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38005ba0 con 0x7fbf2c004850 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4920e700 1 -- 172.21.15.83:0/120932124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2c004850 msgr2=0x7fbf2c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4920e700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2c004850 0x7fbf2c004cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800c210 tx=0x7fbf2c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4920e700 1 -- 172.21.15.83:0/120932124 shutdown_connections 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4920e700 1 --2- 172.21.15.83:0/120932124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2c004850 0x7fbf2c004cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.125+0000 7fbf4920e700 1 -- 172.21.15.83:0/120932124 >> 172.21.15.83:0/120932124 conn(0x7fbf440b2b60 msgr2=0x7fbf440b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.126+0000 7fbf4920e700 1 -- 172.21.15.83:0/120932124 shutdown_connections 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.126+0000 7fbf4920e700 1 -- 172.21.15.83:0/120932124 wait complete. 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.127+0000 7fbf4920e700 1 Processor -- start 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.127+0000 7fbf4920e700 1 -- start start 2023-11-28T11:09:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c004850 0x7fbf4415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 0x7fbf440cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf440ccbe0 0x7fbf4415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441f4250 con 0x7fbf440c6ce0 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440cb210 con 0x7fbf440ccbe0 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441f35d0 con 0x7fbf2c004850 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 0x7fbf440cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 0x7fbf440cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50640/0 (socket says 172.21.15.83:50640) 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 -- 172.21.15.83:0/2115524327 learned_addr learned my addr 172.21.15.83:0/2115524327 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf42d9d700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c004850 0x7fbf4415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4359e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf440ccbe0 0x7fbf4415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 -- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c004850 msgr2=0x7fbf4415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c004850 0x7fbf4415a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 -- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf440ccbe0 msgr2=0x7fbf4415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf440ccbe0 0x7fbf4415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 -- 172.21.15.83:0/2115524327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440b5800 con 0x7fbf440c6ce0 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4259c700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 0x7fbf440cc6a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800c800 tx=0x7fbf38001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38000ed0 con 0x7fbf440c6ce0 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf380070d0 con 0x7fbf440c6ce0 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf440b5520 con 0x7fbf440c6ce0 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.128+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38007460 con 0x7fbf440c6ce0 2023-11-28T11:09:44.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.129+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4406ac30 con 0x7fbf440c6ce0 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.130+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4406d5c0 con 0x7fbf440c6ce0 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.130+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf38024030 con 0x7fbf440c6ce0 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.131+0000 7fbf2b7fe700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf300682f0 0x7fbf3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.131+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fbf38087190 con 0x7fbf440c6ce0 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.131+0000 7fbf42d9d700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf300682f0 0x7fbf3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.131+0000 7fbf42d9d700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf300682f0 0x7fbf3006a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf440c9ce0 tx=0x7fbf34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.135+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf38053ca0 con 0x7fbf440c6ce0 2023-11-28T11:09:44.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.316+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf440b3820 con 0x7fbf440c6ce0 2023-11-28T11:09:44.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.317+0000 7fbf2b7fe700 1 -- 172.21.15.83:0/2115524327 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf38007d70 con 0x7fbf440c6ce0 2023-11-28T11:09:44.318 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf300682f0 msgr2=0x7fbf3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf300682f0 0x7fbf3006a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf440c9ce0 tx=0x7fbf34009040 comp rx=0 tx=0).stop 2023-11-28T11:09:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 msgr2=0x7fbf440cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 0x7fbf440cc6a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800c800 tx=0x7fbf38001a00 comp rx=0 tx=0).stop 2023-11-28T11:09:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 shutdown_connections 2023-11-28T11:09:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf440ccbe0 0x7fbf4415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf2c004850 0x7fbf4415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf300682f0 0x7fbf3006a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 --2- 172.21.15.83:0/2115524327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf440c6ce0 0x7fbf440cc6a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.318+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 >> 172.21.15.83:0/2115524327 conn(0x7fbf440b2b60 msgr2=0x7fbf440b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.319+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 shutdown_connections 2023-11-28T11:09:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.319+0000 7fbf4920e700 1 -- 172.21.15.83:0/2115524327 wait complete. 2023-11-28T11:09:44.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:09:44.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.450+0000 7f832d462700 1 Processor -- start 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.450+0000 7f832d462700 1 -- start start 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f832d462700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f832815a510 0x7f83280cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f832d462700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83280cfdf0 0x7f83280b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f832d462700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832815f1d0 con 0x7f832815a8e0 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f832d462700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83280cd780 con 0x7f83280cfdf0 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f832d462700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832815eef0 con 0x7f832815a510 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f83267fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83280cfdf0 0x7f83280b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f83267fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83280cfdf0 0x7f83280b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50650/0 (socket says 172.21.15.83:50650) 2023-11-28T11:09:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f83267fc700 1 -- 172.21.15.83:0/1861792649 learned_addr learned my addr 172.21.15.83:0/1861792649 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f8326ffd700 1 --2- 172.21.15.83:0/1861792649 >> v2:172.21.15.90:3300/0 conn(0x7f832815a510 0x7f83280cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f83267fc700 1 -- 172.21.15.83:0/1861792649 >> v2:172.21.15.90:3300/0 conn(0x7f832815a510 msgr2=0x7f83280cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f83267fc700 1 --2- 172.21.15.83:0/1861792649 >> v2:172.21.15.90:3300/0 conn(0x7f832815a510 0x7f83280cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.451+0000 7f83267fc700 1 -- 172.21.15.83:0/1861792649 >> v1:172.21.15.90:6789/0 conn(0x7f832815a8e0 legacy=0x7f83280b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f83267fc700 1 -- 172.21.15.83:0/1861792649 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83280cedd0 con 0x7f83280cfdf0 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f8326ffd700 1 --2- 172.21.15.83:0/1861792649 >> v2:172.21.15.90:3300/0 conn(0x7f832815a510 0x7f83280cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f83267fc700 1 --2- 172.21.15.83:0/1861792649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cfdf0 0x7f83280b2960 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f83280ce6b0 tx=0x7f831c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6326d6f5893b716a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f8325ffb700 1 -- 172.21.15.83:0/1861792649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c004230 con 0x7f83280cfdf0 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f8325ffb700 1 -- 172.21.15.83:0/1861792649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c004390 con 0x7f83280cfdf0 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f8325ffb700 1 -- 172.21.15.83:0/1861792649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0036a0 con 0x7f83280cfdf0 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 -- 172.21.15.83:0/1861792649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cfdf0 msgr2=0x7f83280b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 --2- 172.21.15.83:0/1861792649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cfdf0 0x7f83280b2960 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f83280ce6b0 tx=0x7f831c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 -- 172.21.15.83:0/1861792649 shutdown_connections 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 --2- 172.21.15.83:0/1861792649 >> v2:172.21.15.90:3300/0 conn(0x7f832815a510 0x7f83280cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 --2- 172.21.15.83:0/1861792649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cfdf0 0x7f83280b2960 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 -- 172.21.15.83:0/1861792649 >> 172.21.15.83:0/1861792649 conn(0x7f83280cd960 msgr2=0x7f83280cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 -- 172.21.15.83:0/1861792649 shutdown_connections 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.452+0000 7f832d462700 1 -- 172.21.15.83:0/1861792649 wait complete. 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 Processor -- start 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 -- start start 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83280cfdf0 0x7f83281f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 0x7f83281532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f832815a8e0 0x7f83280b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328154680 con 0x7f832815a510 2023-11-28T11:09:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83281f1d90 con 0x7f832815a8e0 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.454+0000 7f832d462700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8328153b00 con 0x7f83280cfdf0 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 0x7f83281532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 0x7f83281532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50654/0 (socket says 172.21.15.83:50654) 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 -- 172.21.15.83:0/4073301389 learned_addr learned my addr 172.21.15.83:0/4073301389 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83277fe700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f832815a8e0 0x7f83280b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f8326ffd700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83280cfdf0 0x7f83281f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 -- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83280cfdf0 msgr2=0x7f83281f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83280cfdf0 0x7f83281f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 -- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f832815a8e0 msgr2=0x7f83280b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f832815a8e0 0x7f83280b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 -- 172.21.15.83:0/4073301389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c009550 con 0x7f832815a510 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f8326ffd700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83280cfdf0 0x7f83281f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f83267fc700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 0x7f83281532b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f83280723f0 tx=0x7f831c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c003cd0 con 0x7f832815a510 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c01f070 con 0x7f832815a510 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83280cf390 con 0x7f832815a510 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c011870 con 0x7f832815a510 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.455+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83280cf0b0 con 0x7f832815a510 2023-11-28T11:09:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.456+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83280b4440 con 0x7f832815a510 2023-11-28T11:09:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.457+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f831c017020 con 0x7f832815a510 2023-11-28T11:09:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.457+0000 7f830f7fe700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f83140682f0 0x7f831406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.457+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f831c087210 con 0x7f832815a510 2023-11-28T11:09:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.457+0000 7f8326ffd700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f83140682f0 0x7f831406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.457+0000 7f8326ffd700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f83140682f0 0x7f831406a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8318006fd0 tx=0x7f8318008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.461+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f831c053c20 con 0x7f832815a510 2023-11-28T11:09:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.646+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8328074320 con 0x7f832815a510 2023-11-28T11:09:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.646+0000 7f830f7fe700 1 -- 172.21.15.83:0/4073301389 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+126 (secure 0 0 0) 0x7f831c057240 con 0x7f832815a510 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": "rbd" 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 2, 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-28T11:09:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:09:44.649 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f83140682f0 msgr2=0x7f831406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f83140682f0 0x7f831406a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8318006fd0 tx=0x7f8318008040 comp rx=0 tx=0).stop 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 msgr2=0x7f83281532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 0x7f83281532b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f83280723f0 tx=0x7f831c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 shutdown_connections 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f832815a8e0 0x7f83280b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83280cfdf0 0x7f83281f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f83140682f0 0x7f831406a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 --2- 172.21.15.83:0/4073301389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815a510 0x7f83281532b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 >> 172.21.15.83:0/4073301389 conn(0x7f83280cd960 msgr2=0x7f83280b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 shutdown_connections 2023-11-28T11:09:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.648+0000 7f832d462700 1 -- 172.21.15.83:0/4073301389 wait complete. 2023-11-28T11:09:44.662 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:09:44.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:09:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.784+0000 7f6927479700 1 Processor -- start 2023-11-28T11:09:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.785+0000 7f6927479700 1 -- start start 2023-11-28T11:09:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.785+0000 7f6927479700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f69200b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.785+0000 7f6927479700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f69200b1e30 0x7f692006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.785+0000 7f6927479700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200cb3a0 con 0x7f69200b5670 2023-11-28T11:09:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.785+0000 7f6927479700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69200b5ae0 con 0x7f69200b1e30 2023-11-28T11:09:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.785+0000 7f6927479700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cb640 con 0x7f69201599b0 2023-11-28T11:09:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6924a14700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f69200b5670 0x7f69200b1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48520/0 (socket says 172.21.15.83:48520) 2023-11-28T11:09:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6924a14700 1 -- 172.21.15.83:0/1643850671 learned_addr learned my addr 172.21.15.83:0/1643850671 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6925215700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f69200b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6925a16700 1 --2- 172.21.15.83:0/1643850671 >> v2:172.21.15.90:3300/0 conn(0x7f69200b1e30 0x7f692006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822565909 0 0) 0x7f69200cb3a0 con 0x7f69200b5670 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f690c0035b0 con 0x7f69200b5670 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2405665246 0 0) 0x7f690c0035b0 con 0x7f69200b5670 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 msgr2=0x7f69200b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f69200b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 >> v2:172.21.15.90:3300/0 conn(0x7f69200b1e30 msgr2=0x7f692006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 --2- 172.21.15.83:0/1643850671 >> v2:172.21.15.90:3300/0 conn(0x7f69200b1e30 0x7f692006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200cb920 con 0x7f69200b5670 2023-11-28T11:09:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f691c001670 con 0x7f69200b5670 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 >> v1:172.21.15.83:6789/0 conn(0x7f69200b5670 legacy=0x7f69200b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690c003a00 0x7f690c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f690c004430 0x7f690c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f690c005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b5ae0 con 0x7f690c003a00 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cb640 con 0x7f690c004430 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c00ba10 con 0x7f69201599b0 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6925215700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690c003a00 0x7f690c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6925a16700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f690c005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.786+0000 7f6924a14700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f690c004430 0x7f690c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925215700 1 -- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 msgr2=0x7f690c005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925215700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f690c005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925215700 1 -- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f690c004430 msgr2=0x7f690c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925215700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f690c004430 0x7f690c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925215700 1 -- 172.21.15.83:0/1643850671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0035b0 con 0x7f690c003a00 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6924a14700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f690c004430 0x7f690c0048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925a16700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69201599b0 0x7f690c005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925215700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690c003a00 0x7f690c003e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6910009b90 tx=0x7f690c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a074a3ba11fe52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000e6c0 con 0x7f690c003a00 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691000e820 con 0x7f690c003a00 2023-11-28T11:09:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6917fff700 1 -- 172.21.15.83:0/1643850671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000eb90 con 0x7f690c003a00 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6927479700 1 -- 172.21.15.83:0/1643850671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690c003a00 msgr2=0x7f690c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6927479700 1 --2- 172.21.15.83:0/1643850671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690c003a00 0x7f690c003e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6910009b90 tx=0x7f690c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6925a16700 1 -- 172.21.15.83:0/1643850671 reap_dead start 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6927479700 1 -- 172.21.15.83:0/1643850671 shutdown_connections 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.787+0000 7f6927479700 1 -- 172.21.15.83:0/1643850671 >> 172.21.15.83:0/1643850671 conn(0x7f69200cc690 msgr2=0x7f69200cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.788+0000 7f6927479700 1 -- 172.21.15.83:0/1643850671 shutdown_connections 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.788+0000 7f6927479700 1 -- 172.21.15.83:0/1643850671 wait complete. 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 Processor -- start 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 -- start start 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f690c003a00 0x7f69200c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 0x7f69200c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6920149380 0x7f69200c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692015cd10 con 0x7f6920148f10 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692015bfe0 con 0x7f6920149380 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.789+0000 7f6927479700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200c8850 con 0x7f690c003a00 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 0x7f69200c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 0x7f69200c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50674/0 (socket says 172.21.15.83:50674) 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 -- 172.21.15.83:0/3540884438 learned_addr learned my addr 172.21.15.83:0/3540884438 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6925a16700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6920149380 0x7f69200c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 -- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f690c003a00 msgr2=0x7f69200c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6925215700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f690c003a00 0x7f69200c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f690c003a00 0x7f69200c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 -- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6920149380 msgr2=0x7f69200c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6920149380 0x7f69200c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 -- 172.21.15.83:0/3540884438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200cbee0 con 0x7f6920148f10 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6924a14700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 0x7f69200c91d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f691c00eaf0 tx=0x7f691c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c005e00 con 0x7f6920148f10 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c00c750 con 0x7f6920148f10 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69200cbc00 con 0x7f6920148f10 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00cb00 con 0x7f6920148f10 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.790+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69200cc1c0 con 0x7f6920148f10 2023-11-28T11:09:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.791+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69200742b0 con 0x7f6920148f10 2023-11-28T11:09:44.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.792+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f691c00cc60 con 0x7f6920148f10 2023-11-28T11:09:44.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.792+0000 7f6915ffb700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f69040682f0 0x7f690406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.792+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f691c0859f0 con 0x7f6920148f10 2023-11-28T11:09:44.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.792+0000 7f6925215700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f69040682f0 0x7f690406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.793+0000 7f6925215700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f69040682f0 0x7f690406a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6910000c00 tx=0x7f6910008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.796+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f691c087e60 con 0x7f6920148f10 2023-11-28T11:09:44.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:44.978+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f69200c34c0 con 0x7f6920148f10 2023-11-28T11:09:45.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.331+0000 7f6915ffb700 1 -- 172.21.15.83:0/3540884438 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f691c052460 con 0x7f6920148f10 2023-11-28T11:09:45.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.334+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f69040682f0 msgr2=0x7f690406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.334+0000 7f6927479700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f69040682f0 0x7f690406a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6910000c00 tx=0x7f6910008030 comp rx=0 tx=0).stop 2023-11-28T11:09:45.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.334+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 msgr2=0x7f69200c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.334+0000 7f6927479700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 0x7f69200c91d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f691c00eaf0 tx=0x7f691c00eeb0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 shutdown_connections 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6920149380 0x7f69200c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f690c003a00 0x7f69200c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f69040682f0 0x7f690406a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 --2- 172.21.15.83:0/3540884438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6920148f10 0x7f69200c91d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 >> 172.21.15.83:0/3540884438 conn(0x7f69200cc690 msgr2=0x7f69200cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 shutdown_connections 2023-11-28T11:09:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.335+0000 7f6927479700 1 -- 172.21.15.83:0/3540884438 wait complete. 2023-11-28T11:09:45.338 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-28T11:09:45.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:09:45.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.475+0000 7f3cee5b4700 1 Processor -- start 2023-11-28T11:09:45.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.475+0000 7f3cee5b4700 1 -- start start 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3cee5b4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ce815ab00 0x7f3ce80c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3cee5b4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce80ca340 0x7f3ce815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3cee5b4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce815e220 con 0x7f3ce806a8a0 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3cee5b4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce815af70 con 0x7f3ce815ab00 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3cee5b4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce815b150 con 0x7f3ce80ca340 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ce815ab00 0x7f3ce80c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3ce806a8a0 0x7f3ce80cb6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48540/0 (socket says 172.21.15.83:48540) 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce7fff700 1 -- 172.21.15.83:0/2646285628 learned_addr learned my addr 172.21.15.83:0/2646285628 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3cecb51700 1 --2- 172.21.15.83:0/2646285628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce80ca340 0x7f3ce815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce77fe700 1 -- 172.21.15.83:0/2646285628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce80ca340 msgr2=0x7f3ce815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce77fe700 1 --2- 172.21.15.83:0/2646285628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce80ca340 0x7f3ce815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce77fe700 1 -- 172.21.15.83:0/2646285628 >> v1:172.21.15.83:6789/0 conn(0x7f3ce806a8a0 legacy=0x7f3ce80cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce6ffd700 1 -- 172.21.15.83:0/2646285628 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016526537 0 0) 0x7f3ce815e220 con 0x7f3ce806a8a0 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce77fe700 1 -- 172.21.15.83:0/2646285628 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce815b430 con 0x7f3ce815ab00 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce77fe700 1 --2- 172.21.15.83:0/2646285628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce815ab00 0x7f3ce80c9e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009840 tx=0x7f3cdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545a72852ac08691 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce6ffd700 1 -- 172.21.15.83:0/2646285628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc011830 con 0x7f3ce815ab00 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce6ffd700 1 -- 172.21.15.83:0/2646285628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc011990 con 0x7f3ce815ab00 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.476+0000 7f3ce6ffd700 1 -- 172.21.15.83:0/2646285628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc011d20 con 0x7f3ce815ab00 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 -- 172.21.15.83:0/2646285628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce815ab00 msgr2=0x7f3ce80c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/2646285628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce815ab00 0x7f3ce80c9e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009840 tx=0x7f3cdc009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 -- 172.21.15.83:0/2646285628 shutdown_connections 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/2646285628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce80ca340 0x7f3ce815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/2646285628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce815ab00 0x7f3ce80c9e00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 -- 172.21.15.83:0/2646285628 >> 172.21.15.83:0/2646285628 conn(0x7f3ce80cf390 msgr2=0x7f3ce80cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 -- 172.21.15.83:0/2646285628 shutdown_connections 2023-11-28T11:09:45.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.477+0000 7f3cee5b4700 1 -- 172.21.15.83:0/2646285628 wait complete. 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.478+0000 7f3cee5b4700 1 Processor -- start 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.478+0000 7f3cee5b4700 1 -- start start 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cee5b4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce806a8a0 0x7f3ce8155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cee5b4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 0x7f3ce80c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cee5b4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce815ab00 0x7f3ce81ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cee5b4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80c37c0 con 0x7f3ce80ca340 2023-11-28T11:09:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cee5b4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80c3aa0 con 0x7f3ce806a8a0 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cee5b4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80c3d80 con 0x7f3ce815ab00 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 0x7f3ce80c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 0x7f3ce80c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50688/0 (socket says 172.21.15.83:50688) 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 -- 172.21.15.83:0/1951657889 learned_addr learned my addr 172.21.15.83:0/1951657889 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3cecb51700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce815ab00 0x7f3ce81ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce7fff700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce806a8a0 0x7f3ce8155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 -- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce815ab00 msgr2=0x7f3ce81ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce815ab00 0x7f3ce81ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 -- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce806a8a0 msgr2=0x7f3ce8155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce806a8a0 0x7f3ce8155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce77fe700 1 -- 172.21.15.83:0/1951657889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0094a0 con 0x7f3ce80ca340 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.479+0000 7f3ce7fff700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce806a8a0 0x7f3ce8155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.480+0000 7f3ce77fe700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 0x7f3ce80c2ec0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc000c00 tx=0x7f3cdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.480+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0036a0 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.480+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc01f070 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.480+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce815b9f0 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.480+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc003a40 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.480+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce815b710 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.481+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce806ac70 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.481+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3cdc00f050 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.481+0000 7f3ce4ff9700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3cd40682f0 0x7f3cd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.482+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f3cdc021070 con 0x7f3ce80ca340 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.482+0000 7f3ce7fff700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3cd40682f0 0x7f3cd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.482+0000 7f3ce7fff700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3cd40682f0 0x7f3cd406a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8000f60 tx=0x7f3cd8007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.486+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3cdc053f70 con 0x7f3ce80ca340 2023-11-28T11:09:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:45.668+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3ce80c4f00 con 0x7f3ce80ca340 2023-11-28T11:09:46.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.332+0000 7f3ce4ff9700 1 -- 172.21.15.83:0/1951657889 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3ce80c4f00 con 0x7f3ce80ca340 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3cd40682f0 msgr2=0x7f3cd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3cd40682f0 0x7f3cd406a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8000f60 tx=0x7f3cd8007d30 comp rx=0 tx=0).stop 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 msgr2=0x7f3ce80c2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 0x7f3ce80c2ec0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc000c00 tx=0x7f3cdc004750 comp rx=0 tx=0).stop 2023-11-28T11:09:46.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 shutdown_connections 2023-11-28T11:09:46.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ce815ab00 0x7f3ce81ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce806a8a0 0x7f3ce8155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3cd40682f0 0x7f3cd406a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 --2- 172.21.15.83:0/1951657889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce80ca340 0x7f3ce80c2ec0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.336+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 >> 172.21.15.83:0/1951657889 conn(0x7f3ce80cf390 msgr2=0x7f3ce80af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.339+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 shutdown_connections 2023-11-28T11:09:46.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.340+0000 7f3cee5b4700 1 -- 172.21.15.83:0/1951657889 wait complete. 2023-11-28T11:09:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-28T11:09:46.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:09:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.486+0000 7ff9f54fb700 1 Processor -- start 2023-11-28T11:09:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9f54fb700 1 -- start start 2023-11-28T11:09:46.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9f54fb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9f0158c50 0x7ff9f00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9f54fb700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f0071a70 0x7ff9f006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9f54fb700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f00c9ea0 con 0x7ff9f00c9620 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9f54fb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f00cab20 con 0x7ff9f0158c50 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9f54fb700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0159020 con 0x7ff9f0071a70 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9eeffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9f0158c50 0x7ff9f00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9eeffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9f0158c50 0x7ff9f00cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50694/0 (socket says 172.21.15.83:50694) 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.487+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1632768437 learned_addr learned my addr 172.21.15.83:0/1632768437 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9ef7fe700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f0071a70 0x7ff9f006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1632768437 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f0071a70 msgr2=0x7ff9f006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9eeffd700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f0071a70 0x7ff9f006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1632768437 >> v1:172.21.15.90:6789/0 conn(0x7ff9f00c9620 legacy=0x7ff9f0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1632768437 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00ca180 con 0x7ff9f0158c50 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9ef7fe700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f0071a70 0x7ff9f006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9eeffd700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0158c50 0x7ff9f00cd670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40098f0 tx=0x7ff9e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba9dca42b04c2bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9edffb700 1 -- 172.21.15.83:0/1632768437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004230 con 0x7ff9f0158c50 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9edffb700 1 -- 172.21.15.83:0/1632768437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e4004390 con 0x7ff9f0158c50 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.488+0000 7ff9edffb700 1 -- 172.21.15.83:0/1632768437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40036a0 con 0x7ff9f0158c50 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1632768437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0158c50 msgr2=0x7ff9f00cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0158c50 0x7ff9f00cd670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40098f0 tx=0x7ff9e4009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1632768437 shutdown_connections 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f0071a70 0x7ff9f006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1632768437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0158c50 0x7ff9f00cd670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1632768437 >> 172.21.15.83:0/1632768437 conn(0x7ff9f00ceb80 msgr2=0x7ff9f00cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1632768437 shutdown_connections 2023-11-28T11:09:46.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.489+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1632768437 wait complete. 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.490+0000 7ff9f54fb700 1 Processor -- start 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.490+0000 7ff9f54fb700 1 -- start start 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9f54fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 0x7ff9f00affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9f54fb700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f00c9620 0x7ff9f0151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9f54fb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9f0158c50 0x7ff9f006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9f54fb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00b3530 con 0x7ff9f0071a70 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9f54fb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00b0500 con 0x7ff9f0158c50 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9f54fb700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00b2c20 con 0x7ff9f00c9620 2023-11-28T11:09:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 0x7ff9f00affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 0x7ff9f00affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50700/0 (socket says 172.21.15.83:50700) 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1331501198 learned_addr learned my addr 172.21.15.83:0/1331501198 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9ee7fc700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f00c9620 0x7ff9f0151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9ef7fe700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9f0158c50 0x7ff9f006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f00c9620 msgr2=0x7ff9f0151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f00c9620 0x7ff9f0151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9f0158c50 msgr2=0x7ff9f006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9f0158c50 0x7ff9f006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 -- 172.21.15.83:0/1331501198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e4009550 con 0x7ff9f0071a70 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9eeffd700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 0x7ff9f00affc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff9f006eda0 tx=0x7ff9e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4003e90 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.491+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e400b660 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.492+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f00ca740 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.492+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400b9f0 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.492+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f00ca460 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.492+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f00b3670 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.493+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff9e4017020 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.493+0000 7ff9d77fe700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff9dc0682f0 0x7ff9dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.493+0000 7ff9ee7fc700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff9dc0682f0 0x7ff9dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.494+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6282+0+0 (secure 0 0 0) 0x7ff9e40874e0 con 0x7ff9f0071a70 2023-11-28T11:09:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.494+0000 7ff9ee7fc700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff9dc0682f0 0x7ff9dc06a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0157a20 tx=0x7ff9e0007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.498+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff9f00b3670 con 0x7ff9f0071a70 2023-11-28T11:09:46.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.726+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff9f00c9aa0 con 0x7ff9f0071a70 2023-11-28T11:09:46.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.731+0000 7ff9d77fe700 1 -- 172.21.15.83:0/1331501198 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff9e4053b10 con 0x7ff9f0071a70 2023-11-28T11:09:46.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff9dc0682f0 msgr2=0x7ff9dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff9dc0682f0 0x7ff9dc06a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0157a20 tx=0x7ff9e0007ec0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 msgr2=0x7ff9f00affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 0x7ff9f00affc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff9f006eda0 tx=0x7ff9e4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 shutdown_connections 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9f0158c50 0x7ff9f006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff9dc0682f0 0x7ff9dc06a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f0071a70 0x7ff9f00affc0 secure :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff9f006eda0 tx=0x7ff9e4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 --2- 172.21.15.83:0/1331501198 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff9f00c9620 0x7ff9f0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.735+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 >> 172.21.15.83:0/1331501198 conn(0x7ff9f00ceb80 msgr2=0x7ff9f015ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.737+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 shutdown_connections 2023-11-28T11:09:46.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.741+0000 7ff9f54fb700 1 -- 172.21.15.83:0/1331501198 wait complete. 2023-11-28T11:09:46.749 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 6 and data pool 7 2023-11-28T11:09:46.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:46.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.884+0000 7fab0377f700 1 Processor -- start 2023-11-28T11:09:46.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.884+0000 7fab0377f700 1 -- start start 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab0377f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faafc15cd10 0x7faafc0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab0377f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc0cbfb0 con 0x7faafc0b1fb0 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab0377f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc0cb710 con 0x7faafc15d0e0 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab0377f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faafc0cb9f0 con 0x7faafc15cd10 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab0151b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faafc15cd10 0x7faafc0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab01d1c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faafc0b1fb0 0x7faafc06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48572/0 (socket says 172.21.15.83:48572) 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.885+0000 7fab01d1c700 1 -- 172.21.15.83:0/1740283038 learned_addr learned my addr 172.21.15.83:0/1740283038 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0151b700 1 -- 172.21.15.83:0/1740283038 >> v1:172.21.15.90:6789/0 conn(0x7faafc15d0e0 legacy=0x7faafc0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0151b700 1 -- 172.21.15.83:0/1740283038 >> v1:172.21.15.83:6789/0 conn(0x7faafc0b1fb0 legacy=0x7faafc06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7faaf3fff700 1 -- 172.21.15.83:0/1740283038 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897238457 0 0) 0x7faafc0cbfb0 con 0x7faafc0b1fb0 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0151b700 1 -- 172.21.15.83:0/1740283038 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0cbcd0 con 0x7faafc15cd10 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0151b700 1 --2- 172.21.15.83:0/1740283038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc0ca970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faaec009840 tx=0x7faaec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36507476df12ae8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7faaf3fff700 1 -- 172.21.15.83:0/1740283038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec004230 con 0x7faafc15cd10 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7faaf3fff700 1 -- 172.21.15.83:0/1740283038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec004390 con 0x7faafc15cd10 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7faaf3fff700 1 -- 172.21.15.83:0/1740283038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec0036a0 con 0x7faafc15cd10 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0377f700 1 -- 172.21.15.83:0/1740283038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 msgr2=0x7faafc0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0377f700 1 --2- 172.21.15.83:0/1740283038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc0ca970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faaec009840 tx=0x7faaec009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0377f700 1 -- 172.21.15.83:0/1740283038 shutdown_connections 2023-11-28T11:09:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0377f700 1 --2- 172.21.15.83:0/1740283038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc0ca970 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.886+0000 7fab0377f700 1 -- 172.21.15.83:0/1740283038 >> 172.21.15.83:0/1740283038 conn(0x7faafc06e9a0 msgr2=0x7faafc0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.887+0000 7fab0377f700 1 -- 172.21.15.83:0/1740283038 shutdown_connections 2023-11-28T11:09:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.887+0000 7fab0377f700 1 -- 172.21.15.83:0/1740283038 wait complete. 2023-11-28T11:09:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 Processor -- start 2023-11-28T11:09:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 -- start start 2023-11-28T11:09:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc0b1fb0 0x7faafc15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc15f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faafc15d0e0 0x7faafc1f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0ab8b0 con 0x7faafc15cd10 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0ac710 con 0x7faafc0b1fb0 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.888+0000 7fab0377f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0abb00 con 0x7faafc15d0e0 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc15f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50716/0 (socket says 172.21.15.83:50716) 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 -- 172.21.15.83:0/653825926 learned_addr learned my addr 172.21.15.83:0/653825926 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab01d1c700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faafc15d0e0 0x7faafc1f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab0151b700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc0b1fb0 0x7faafc15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 -- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faafc15d0e0 msgr2=0x7faafc1f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faafc15d0e0 0x7faafc1f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 -- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc0b1fb0 msgr2=0x7faafc15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc0b1fb0 0x7faafc15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 -- 172.21.15.83:0/653825926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec0094a0 con 0x7faafc15cd10 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab01d1c700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faafc15d0e0 0x7faafc1f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab00d1a700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc15f270 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faaf800b470 tx=0x7faaf800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011830 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf8011990 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc06b660 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011d40 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.889+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc06b9b0 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.890+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc0cd9f0 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.891+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faaf8021070 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.891+0000 7faaf1ffb700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faae00682f0 0x7faae006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.891+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faaf8085cd0 con 0x7faafc15cd10 2023-11-28T11:09:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.891+0000 7fab0151b700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faae00682f0 0x7faae006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.892+0000 7fab0151b700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faae00682f0 0x7faae006a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faafc0b30b0 tx=0x7faaec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:46.895+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faaf8052390 con 0x7faafc15cd10 2023-11-28T11:09:47.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.078+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faafc0ac880 con 0x7faafc15cd10 2023-11-28T11:09:47.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.080+0000 7faaf1ffb700 1 -- 172.21.15.83:0/653825926 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18035 (secure 0 0 0) 0x7faaf8015020 con 0x7faafc15cd10 2023-11-28T11:09:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:47.081 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:46.728058+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:47.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faae00682f0 msgr2=0x7faae006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faae00682f0 0x7faae006a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faafc0b30b0 tx=0x7faaec008030 comp rx=0 tx=0).stop 2023-11-28T11:09:47.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 msgr2=0x7faafc15f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc15f270 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faaf800b470 tx=0x7faaf800b830 comp rx=0 tx=0).stop 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 shutdown_connections 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faafc15d0e0 0x7faafc1f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc0b1fb0 0x7faafc15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7faae00682f0 0x7faae006a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 --2- 172.21.15.83:0/653825926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faafc15cd10 0x7faafc15f270 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 >> 172.21.15.83:0/653825926 conn(0x7faafc06e9a0 msgr2=0x7faafc0afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 shutdown_connections 2023-11-28T11:09:47.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.082+0000 7fab0377f700 1 -- 172.21.15.83:0/653825926 wait complete. 2023-11-28T11:09:47.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:47.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.214+0000 7f9467092700 1 Processor -- start 2023-11-28T11:09:47.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.214+0000 7f9467092700 1 -- start start 2023-11-28T11:09:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.214+0000 7f9467092700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f94600cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f9467092700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946015efa0 con 0x7f94600cfce0 2023-11-28T11:09:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f9467092700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600cecc0 con 0x7f946015a850 2023-11-28T11:09:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f9467092700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600ceab0 con 0x7f946015a480 2023-11-28T11:09:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f946562f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f94600cfce0 0x7f94600b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48586/0 (socket says 172.21.15.83:48586) 2023-11-28T11:09:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f946562f700 1 -- 172.21.15.83:0/508843438 learned_addr learned my addr 172.21.15.83:0/508843438 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f9464e2e700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f94600cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730986127 0 0) 0x7f946015efa0 con 0x7f94600cfce0 2023-11-28T11:09:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94500035b0 con 0x7f94600cfce0 2023-11-28T11:09:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303288265 0 0) 0x7f94600cecc0 con 0x7f946015a850 2023-11-28T11:09:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946015efa0 con 0x7f946015a850 2023-11-28T11:09:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126430528 0 0) 0x7f94500035b0 con 0x7f94600cfce0 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 msgr2=0x7f94600cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f94600cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 >> v1:172.21.15.90:6789/0 conn(0x7f946015a850 legacy=0x7f94600b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600cefa0 con 0x7f94600cfce0 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f9464e2e700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f94600cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f94580019f0 con 0x7f94600cfce0 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 >> v1:172.21.15.83:6789/0 conn(0x7f94600cfce0 legacy=0x7f94600b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450003bc0 0x7f9450003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9450004850 0x7f9450004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f9450005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600ceab0 con 0x7f9450004850 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945000bac0 con 0x7f9450003bc0 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945000bc80 con 0x7f946015a480 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.215+0000 7f945ffff700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9450004850 0x7f9450004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f9464e2e700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450003bc0 0x7f9450003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f946562f700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f9450005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945ffff700 1 -- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 msgr2=0x7f9450005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945ffff700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f9450005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945ffff700 1 -- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450003bc0 msgr2=0x7f9450003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945ffff700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450003bc0 0x7f9450003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945ffff700 1 -- 172.21.15.83:0/508843438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540094a0 con 0x7f9450004850 2023-11-28T11:09:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f946562f700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f946015a480 0x7f9450005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945ffff700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9450004850 0x7f9450004cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f944c001940 tx=0x7f9450005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276008998629b48f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c001430 con 0x7f9450004850 2023-11-28T11:09:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c001d70 con 0x7f9450004850 2023-11-28T11:09:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f945f7fe700 1 -- 172.21.15.83:0/508843438 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c005510 con 0x7f9450004850 2023-11-28T11:09:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f9467092700 1 -- 172.21.15.83:0/508843438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9450004850 msgr2=0x7f9450004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f9467092700 1 --2- 172.21.15.83:0/508843438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9450004850 0x7f9450004cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f944c001940 tx=0x7f9450005880 comp rx=0 tx=0).stop 2023-11-28T11:09:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f946562f700 1 -- 172.21.15.83:0/508843438 reap_dead start 2023-11-28T11:09:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f9467092700 1 -- 172.21.15.83:0/508843438 shutdown_connections 2023-11-28T11:09:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.216+0000 7f9467092700 1 -- 172.21.15.83:0/508843438 >> 172.21.15.83:0/508843438 conn(0x7f946006de20 msgr2=0x7f946006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.217+0000 7f9467092700 1 -- 172.21.15.83:0/508843438 shutdown_connections 2023-11-28T11:09:47.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.217+0000 7f9467092700 1 -- 172.21.15.83:0/508843438 wait complete. 2023-11-28T11:09:47.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.218+0000 7f9467092700 1 Processor -- start 2023-11-28T11:09:47.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.218+0000 7f9467092700 1 -- start start 2023-11-28T11:09:47.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9450004850 0x7f946015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 0x7f94600c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94600b5990 0x7f94600b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600c7880 con 0x7f94600c6ef0 2023-11-28T11:09:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b6340 con 0x7f94600b5990 2023-11-28T11:09:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946015ad30 con 0x7f9450004850 2023-11-28T11:09:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 0x7f94600c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 0x7f94600c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50736/0 (socket says 172.21.15.83:50736) 2023-11-28T11:09:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 -- 172.21.15.83:0/346202492 learned_addr learned my addr 172.21.15.83:0/346202492 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9464e2e700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9450004850 0x7f946015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f946562f700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94600b5990 0x7f94600b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 -- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9450004850 msgr2=0x7f946015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9450004850 0x7f946015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 -- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94600b5990 msgr2=0x7f94600b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94600b5990 0x7f94600b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 -- 172.21.15.83:0/346202492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600b4440 con 0x7f94600c6ef0 2023-11-28T11:09:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9464e2e700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9450004850 0x7f946015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f946562f700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94600b5990 0x7f94600b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945ffff700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 0x7f94600c7340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f944c005580 tx=0x7f944c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c006c70 con 0x7f94600c6ef0 2023-11-28T11:09:47.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c006dd0 con 0x7f94600c6ef0 2023-11-28T11:09:47.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600cf280 con 0x7f94600c6ef0 2023-11-28T11:09:47.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c007280 con 0x7f94600c6ef0 2023-11-28T11:09:47.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.219+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94600b45d0 con 0x7f94600c6ef0 2023-11-28T11:09:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.220+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94601f3290 con 0x7f94600c6ef0 2023-11-28T11:09:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.221+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f944c02d070 con 0x7f94600c6ef0 2023-11-28T11:09:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.221+0000 7f945d7fa700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9444068320 0x7f944406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.221+0000 7f9464e2e700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9444068320 0x7f944406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.221+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f944c086b60 con 0x7f94600c6ef0 2023-11-28T11:09:47.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.222+0000 7f9464e2e700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9444068320 0x7f944406a7d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9454009c10 tx=0x7f9454008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.225+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f944c053280 con 0x7f94600c6ef0 2023-11-28T11:09:47.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.409+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9460074320 con 0x7f94600c6ef0 2023-11-28T11:09:47.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.410+0000 7f945d7fa700 1 -- 172.21.15.83:0/346202492 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18035 (secure 0 0 0) 0x7f944c0568a0 con 0x7f94600c6ef0 2023-11-28T11:09:47.412 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:47.412 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:46.728058+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:47.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9444068320 msgr2=0x7f944406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9444068320 0x7f944406a7d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9454009c10 tx=0x7f9454008040 comp rx=0 tx=0).stop 2023-11-28T11:09:47.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 msgr2=0x7f94600c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 0x7f94600c7340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f944c005580 tx=0x7f944c0012d0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 shutdown_connections 2023-11-28T11:09:47.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94600b5990 0x7f94600b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9450004850 0x7f946015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9444068320 0x7f944406a7d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 --2- 172.21.15.83:0/346202492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94600c6ef0 0x7f94600c7340 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.412+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 >> 172.21.15.83:0/346202492 conn(0x7f946006de20 msgr2=0x7f94600cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.413+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 shutdown_connections 2023-11-28T11:09:47.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.413+0000 7f9467092700 1 -- 172.21.15.83:0/346202492 wait complete. 2023-11-28T11:09:47.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:09:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.548+0000 7fcbe2d9d700 1 Processor -- start 2023-11-28T11:09:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.548+0000 7fcbe2d9d700 1 -- start start 2023-11-28T11:09:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe2d9d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15c890 0x7fcbdc0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe2d9d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcbdc06e370 0x7fcbdc06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe2d9d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbdc06d4a0 con 0x7fcbdc0b4bb0 2023-11-28T11:09:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe2d9d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b5020 con 0x7fcbdc06e370 2023-11-28T11:09:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe2d9d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b5ae0 con 0x7fcbdc15c890 2023-11-28T11:09:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdbfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcbdc0b4bb0 0x7fcbdc06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48602/0 (socket says 172.21.15.83:48602) 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdbfff700 1 -- 172.21.15.83:0/3589611401 learned_addr learned my addr 172.21.15.83:0/3589611401 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3589611401 >> v2:172.21.15.90:3300/0 conn(0x7fcbdc06e370 0x7fcbdc06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbe0b39700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15c890 0x7fcbdc0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263536681 0 0) 0x7fcbdc06d4a0 con 0x7fcbdc0b4bb0 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbcc0035b0 con 0x7fcbdc0b4bb0 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694435552 0 0) 0x7fcbcc0035b0 con 0x7fcbdc0b4bb0 2023-11-28T11:09:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15c890 msgr2=0x7fcbdc0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15c890 0x7fcbdc0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 >> v2:172.21.15.90:3300/0 conn(0x7fcbdc06e370 msgr2=0x7fcbdc06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 --2- 172.21.15.83:0/3589611401 >> v2:172.21.15.90:3300/0 conn(0x7fcbdc06e370 0x7fcbdc06b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc06c400 con 0x7fcbdc0b4bb0 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.549+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fcbc8001670 con 0x7fcbdc0b4bb0 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 >> v1:172.21.15.83:6789/0 conn(0x7fcbdc0b4bb0 legacy=0x7fcbdc06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbcc003a00 0x7fcbcc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbcc004430 0x7fcbcc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc004fb0 0x7fcbcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b5020 con 0x7fcbcc004fb0 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b5ae0 con 0x7fcbcc003a00 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc00bf50 con 0x7fcbcc004430 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe0b39700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15c890 0x7fcbdc0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3589611401 >> v2:172.21.15.90:3300/0 conn(0x7fcbdc06e370 0x7fcbdc06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc004fb0 0x7fcbcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe0b39700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbcc003a00 0x7fcbcc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdbfff700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbcc004430 0x7fcbcc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 -- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbcc004430 msgr2=0x7fcbcc0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbcc004430 0x7fcbcc0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 -- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbcc003a00 msgr2=0x7fcbcc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbcc003a00 0x7fcbcc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 -- 172.21.15.83:0/3589611401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0035b0 con 0x7fcbcc004fb0 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc004fb0 0x7fcbcc00ba10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400e350 tx=0x7fcbcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412df022592a1948 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400ee10 con 0x7fcbcc004fb0 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd400f040 con 0x7fcbcc004fb0 2023-11-28T11:09:47.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.550+0000 7fcbdb7fe700 1 -- 172.21.15.83:0/3589611401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4003c40 con 0x7fcbcc004fb0 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3589611401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc004fb0 msgr2=0x7fcbcc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3589611401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc004fb0 0x7fcbcc00ba10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400e350 tx=0x7fcbcc005880 comp rx=0 tx=0).stop 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe133a700 1 -- 172.21.15.83:0/3589611401 reap_dead start 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3589611401 shutdown_connections 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3589611401 >> 172.21.15.83:0/3589611401 conn(0x7fcbdc0b2b60 msgr2=0x7fcbdc0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3589611401 shutdown_connections 2023-11-28T11:09:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.551+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3589611401 wait complete. 2023-11-28T11:09:47.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.552+0000 7fcbe2d9d700 1 Processor -- start 2023-11-28T11:09:47.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.552+0000 7fcbe2d9d700 1 -- start start 2023-11-28T11:09:47.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe2d9d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15aa50 0x7fcbdc09a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe2d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc15ae20 0x7fcbdc15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 0x7fcbdc0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe2d9d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc09aae0 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe2d9d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c41f0 con 0x7fcbdc15ae20 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe2d9d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c4490 con 0x7fcbdc15aa50 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 0x7fcbdc0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 0x7fcbdc0cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50756/0 (socket says 172.21.15.83:50756) 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 -- 172.21.15.83:0/3546604459 learned_addr learned my addr 172.21.15.83:0/3546604459 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbdbfff700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc15ae20 0x7fcbdc15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe0b39700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15aa50 0x7fcbdc09a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 -- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15aa50 msgr2=0x7fcbdc09a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15aa50 0x7fcbdc09a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 -- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc15ae20 msgr2=0x7fcbdc15efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc15ae20 0x7fcbdc15efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.553+0000 7fcbe133a700 1 -- 172.21.15.83:0/3546604459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0b5520 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.554+0000 7fcbe133a700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 0x7fcbdc0cbf30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4004000 tx=0x7fcbd4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.554+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004440 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.554+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd40045c0 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.554+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc0b5240 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.554+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004970 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.554+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc0b5800 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.555+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc15fec0 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.555+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbd401a020 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.555+0000 7fcbd97fa700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcbc00682f0 0x7fcbc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.555+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcbd4088120 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.556+0000 7fcbe0b39700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcbc00682f0 0x7fcbc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.556+0000 7fcbe0b39700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcbc00682f0 0x7fcbc006a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0715b0 tx=0x7fcbd0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.560+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbd40548f0 con 0x7fcbdc15b2f0 2023-11-28T11:09:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:47.790+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcbdc0742b0 con 0x7fcbdc15b2f0 2023-11-28T11:09:48.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.740+0000 7fcbd97fa700 1 -- 172.21.15.83:0/3546604459 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcbd4057f10 con 0x7fcbdc15b2f0 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcbc00682f0 msgr2=0x7fcbc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcbc00682f0 0x7fcbc006a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0715b0 tx=0x7fcbd0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 msgr2=0x7fcbdc0cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 0x7fcbdc0cbf30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4004000 tx=0x7fcbd4009750 comp rx=0 tx=0).stop 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 shutdown_connections 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcbc00682f0 0x7fcbc006a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbdc15b2f0 0x7fcbdc0cbf30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbdc15aa50 0x7fcbdc09a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.743+0000 7fcbe2d9d700 1 --2- 172.21.15.83:0/3546604459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc15ae20 0x7fcbdc15efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.744+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 >> 172.21.15.83:0/3546604459 conn(0x7fcbdc0b2b60 msgr2=0x7fcbdc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.744+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 shutdown_connections 2023-11-28T11:09:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.744+0000 7fcbe2d9d700 1 -- 172.21.15.83:0/3546604459 wait complete. 2023-11-28T11:09:48.762 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:09:48.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:09:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.889+0000 7f50645db700 1 Processor -- start 2023-11-28T11:09:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.889+0000 7f50645db700 1 -- start start 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f50645db700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f505c0af1f0 0x7f505c0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f50645db700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c06c800 con 0x7f505c0aee20 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f50645db700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c06cae0 con 0x7f505c06c370 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f50645db700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c06cdc0 con 0x7f505c0af1f0 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062b78700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f505c0af1f0 0x7f505c0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062377700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f505c0aee20 0x7f505c06ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48620/0 (socket says 172.21.15.83:48620) 2023-11-28T11:09:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062377700 1 -- 172.21.15.83:0/1678372722 learned_addr learned my addr 172.21.15.83:0/1678372722 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062b78700 1 -- 172.21.15.83:0/1678372722 >> v1:172.21.15.90:6789/0 conn(0x7f505c06c370 legacy=0x7f505c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062b78700 1 -- 172.21.15.83:0/1678372722 >> v1:172.21.15.83:6789/0 conn(0x7f505c0aee20 legacy=0x7f505c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5061375700 1 -- 172.21.15.83:0/1678372722 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126815879 0 0) 0x7f505c06c800 con 0x7f505c0aee20 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062b78700 1 -- 172.21.15.83:0/1678372722 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c06d0a0 con 0x7f505c0af1f0 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5062b78700 1 --2- 172.21.15.83:0/1678372722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c0af1f0 0x7f505c0d0710 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5054009d90 tx=0x7f505400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14d6a9eddf2f205 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5061375700 1 -- 172.21.15.83:0/1678372722 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400e070 con 0x7f505c0af1f0 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5061375700 1 -- 172.21.15.83:0/1678372722 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505400bb90 con 0x7f505c0af1f0 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.891+0000 7f5061375700 1 -- 172.21.15.83:0/1678372722 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054003a20 con 0x7f505c0af1f0 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 -- 172.21.15.83:0/1678372722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c0af1f0 msgr2=0x7f505c0d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 --2- 172.21.15.83:0/1678372722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c0af1f0 0x7f505c0d0710 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5054009d90 tx=0x7f505400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 -- 172.21.15.83:0/1678372722 shutdown_connections 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 --2- 172.21.15.83:0/1678372722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c0af1f0 0x7f505c0d0710 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 -- 172.21.15.83:0/1678372722 >> 172.21.15.83:0/1678372722 conn(0x7f505c0b2b60 msgr2=0x7f505c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 -- 172.21.15.83:0/1678372722 shutdown_connections 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.892+0000 7f50645db700 1 -- 172.21.15.83:0/1678372722 wait complete. 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.893+0000 7f50645db700 1 Processor -- start 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.893+0000 7f50645db700 1 -- start start 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f50645db700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 0x7f505c1f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f50645db700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505c0aee20 0x7f505c0adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f50645db700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505c0af1f0 0x7f505c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f50645db700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0ae220 con 0x7f505c06c370 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f50645db700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0acc40 con 0x7f505c0af1f0 2023-11-28T11:09:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f50645db700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0acf20 con 0x7f505c0aee20 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 0x7f505c1f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 0x7f505c1f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50766/0 (socket says 172.21.15.83:50766) 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 -- 172.21.15.83:0/1319756718 learned_addr learned my addr 172.21.15.83:0/1319756718 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062b78700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505c0af1f0 0x7f505c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5061b76700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505c0aee20 0x7f505c0adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 -- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505c0aee20 msgr2=0x7f505c0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505c0aee20 0x7f505c0adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 -- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505c0af1f0 msgr2=0x7f505c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505c0af1f0 0x7f505c1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062377700 1 -- 172.21.15.83:0/1319756718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054009a70 con 0x7f505c06c370 2023-11-28T11:09:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5061b76700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505c0aee20 0x7f505c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.894+0000 7f5062b78700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505c0af1f0 0x7f505c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.895+0000 7f5062377700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 0x7f505c1f51e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f504c00c700 tx=0x7f504c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.895+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0074e0 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.895+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c007640 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.895+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c06d380 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.895+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c005a30 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.895+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c06fc70 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.896+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c06f490 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.896+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f504c019030 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.897+0000 7f5052ffd700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50400682f0 0x7f504006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.897+0000 7f5061b76700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50400682f0 0x7f504006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.897+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f504c0851e0 con 0x7f505c06c370 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.897+0000 7f5061b76700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50400682f0 0x7f504006a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f505c06b7f0 tx=0x7f5058009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:48.901+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f504c04b5e0 con 0x7f505c06c370 2023-11-28T11:09:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.126+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f505c0ceb10 con 0x7f505c06c370 2023-11-28T11:09:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.746+0000 7f5052ffd700 1 -- 172.21.15.83:0/1319756718 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f504c0518c0 con 0x7f505c06c370 2023-11-28T11:09:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50400682f0 msgr2=0x7f504006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50400682f0 0x7f504006a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f505c06b7f0 tx=0x7f5058009040 comp rx=0 tx=0).stop 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 msgr2=0x7f505c1f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 0x7f505c1f51e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f504c00c700 tx=0x7f504c00cac0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 shutdown_connections 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505c0af1f0 0x7f505c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50400682f0 0x7f504006a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f505c06c370 0x7f505c1f51e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 --2- 172.21.15.83:0/1319756718 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f505c0aee20 0x7f505c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 >> 172.21.15.83:0/1319756718 conn(0x7f505c0b2b60 msgr2=0x7f505c0cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 shutdown_connections 2023-11-28T11:09:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.749+0000 7f50645db700 1 -- 172.21.15.83:0/1319756718 wait complete. 2023-11-28T11:09:49.764 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:09:49.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.886+0000 7f31c3298700 1 Processor -- start 2023-11-28T11:09:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.886+0000 7f31c3298700 1 -- start start 2023-11-28T11:09:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c3298700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a510 0x7f31bc0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c3298700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f31bc0cfdf0 0x7f31bc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c3298700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31bc15a8e0 0x7f31bc0b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c3298700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc0cd780 con 0x7f31bc15a8e0 2023-11-28T11:09:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c3298700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc15f1d0 con 0x7f31bc0cfdf0 2023-11-28T11:09:49.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c3298700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc15eef0 con 0x7f31bc15a510 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c1835700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31bc15a8e0 0x7f31bc0b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c1835700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31bc15a8e0 0x7f31bc0b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50776/0 (socket says 172.21.15.83:50776) 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c1835700 1 -- 172.21.15.83:0/1076973088 learned_addr learned my addr 172.21.15.83:0/1076973088 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c0833700 1 --2- 172.21.15.83:0/1076973088 >> v2:172.21.15.90:3300/0 conn(0x7f31bc0cfdf0 0x7f31bc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.887+0000 7f31c1034700 1 --2- 172.21.15.83:0/1076973088 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a510 0x7f31bc0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c1835700 1 -- 172.21.15.83:0/1076973088 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a510 msgr2=0x7f31bc0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c1835700 1 --2- 172.21.15.83:0/1076973088 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a510 0x7f31bc0cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c1835700 1 -- 172.21.15.83:0/1076973088 >> v2:172.21.15.90:3300/0 conn(0x7f31bc0cfdf0 msgr2=0x7f31bc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c1835700 1 --2- 172.21.15.83:0/1076973088 >> v2:172.21.15.90:3300/0 conn(0x7f31bc0cfdf0 0x7f31bc0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c1835700 1 -- 172.21.15.83:0/1076973088 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc0cedd0 con 0x7f31bc15a8e0 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c1835700 1 --2- 172.21.15.83:0/1076973088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a8e0 0x7f31bc0b2ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f31bc072750 tx=0x7f31b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5217c4ace484bbd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31b3fff700 1 -- 172.21.15.83:0/1076973088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b400e070 con 0x7f31bc15a8e0 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31b3fff700 1 -- 172.21.15.83:0/1076973088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b400b9c0 con 0x7f31bc15a8e0 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31b3fff700 1 -- 172.21.15.83:0/1076973088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b400bd70 con 0x7f31bc15a8e0 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 -- 172.21.15.83:0/1076973088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a8e0 msgr2=0x7f31bc0b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 --2- 172.21.15.83:0/1076973088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a8e0 0x7f31bc0b2ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f31bc072750 tx=0x7f31b400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 -- 172.21.15.83:0/1076973088 shutdown_connections 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 --2- 172.21.15.83:0/1076973088 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a510 0x7f31bc0cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 --2- 172.21.15.83:0/1076973088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a8e0 0x7f31bc0b2ea0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 --2- 172.21.15.83:0/1076973088 >> v2:172.21.15.90:3300/0 conn(0x7f31bc0cfdf0 0x7f31bc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.888+0000 7f31c3298700 1 -- 172.21.15.83:0/1076973088 >> 172.21.15.83:0/1076973088 conn(0x7f31bc0cd960 msgr2=0x7f31bc0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.889+0000 7f31c3298700 1 -- 172.21.15.83:0/1076973088 shutdown_connections 2023-11-28T11:09:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.889+0000 7f31c3298700 1 -- 172.21.15.83:0/1076973088 wait complete. 2023-11-28T11:09:49.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 Processor -- start 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 -- start start 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31bc0cfdf0 0x7f31bc157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 0x7f31bc0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a8e0 0x7f31bc1f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc157a90 con 0x7f31bc15a510 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0cb0c0 con 0x7f31bc0cfdf0 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c3298700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0cba90 con 0x7f31bc15a8e0 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c0833700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 0x7f31bc0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c0833700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 0x7f31bc0cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50780/0 (socket says 172.21.15.83:50780) 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.890+0000 7f31c0833700 1 -- 172.21.15.83:0/1331413972 learned_addr learned my addr 172.21.15.83:0/1331413972 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c1034700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31bc0cfdf0 0x7f31bc157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c1835700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a8e0 0x7f31bc1f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c0833700 1 -- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a8e0 msgr2=0x7f31bc1f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c0833700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a8e0 0x7f31bc1f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c0833700 1 -- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31bc0cfdf0 msgr2=0x7f31bc157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c0833700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31bc0cfdf0 0x7f31bc157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c0833700 1 -- 172.21.15.83:0/1331413972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4009b30 con 0x7f31bc15a510 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c0833700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 0x7f31bc0cb550 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f31b800e840 tx=0x7f31b800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b800c070 con 0x7f31bc15a510 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b8013610 con 0x7f31bc15a510 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc0cf390 con 0x7f31bc15a510 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b80139a0 con 0x7f31bc15a510 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.891+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc0cf0b0 con 0x7f31bc15a510 2023-11-28T11:09:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.892+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc0b3d30 con 0x7f31bc15a510 2023-11-28T11:09:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.892+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f31b8023070 con 0x7f31bc15a510 2023-11-28T11:09:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.893+0000 7f31b1ffb700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31a00682f0 0x7f31a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.893+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f31b8085a00 con 0x7f31bc15a510 2023-11-28T11:09:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.893+0000 7f31c1034700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31a00682f0 0x7f31a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.893+0000 7f31c1034700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31a00682f0 0x7f31a006a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f31ac006b20 tx=0x7f31ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:49.897+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f31b8052120 con 0x7f31bc15a510 2023-11-28T11:09:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.124+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31bc0cb270 con 0x7f31bc15a510 2023-11-28T11:09:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.124+0000 7f31b1ffb700 1 -- 172.21.15.83:0/1331413972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f31b8059230 con 0x7f31bc15a510 2023-11-28T11:09:50.126 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:50.126 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31a00682f0 msgr2=0x7f31a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31a00682f0 0x7f31a006a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f31ac006b20 tx=0x7f31ac008040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 msgr2=0x7f31bc0cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 0x7f31bc0cb550 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f31b800e840 tx=0x7f31b800eb50 comp rx=0 tx=0).stop 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 shutdown_connections 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f31bc15a8e0 0x7f31bc1f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31bc0cfdf0 0x7f31bc157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f31a00682f0 0x7f31a006a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 --2- 172.21.15.83:0/1331413972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31bc15a510 0x7f31bc0cb550 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 >> 172.21.15.83:0/1331413972 conn(0x7f31bc0cd960 msgr2=0x7f31bc0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 shutdown_connections 2023-11-28T11:09:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.126+0000 7f31c3298700 1 -- 172.21.15.83:0/1331413972 wait complete. 2023-11-28T11:09:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:09:50.137 INFO:tasks.ceph.mds.d.smithi090.stderr:2023-11-28T11:09:50.136+0000 7f02c72f5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:09:50.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.258+0000 7fbfbefc5700 1 Processor -- start 2023-11-28T11:09:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.258+0000 7fbfbefc5700 1 -- start start 2023-11-28T11:09:50.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.258+0000 7fbfbefc5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfb815cd10 0x7fbfb80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.258+0000 7fbfbefc5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb80b1fb0 0x7fbfb80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbefc5700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb80cbfb0 con 0x7fbfb815d0e0 2023-11-28T11:09:50.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbefc5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb806cee0 con 0x7fbfb815cd10 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbefc5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80cb710 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfb815cd10 0x7fbfb80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfb815cd10 0x7fbfb80ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50786/0 (socket says 172.21.15.83:50786) 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 -- 172.21.15.83:0/964828600 learned_addr learned my addr 172.21.15.83:0/964828600 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbd562700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb80b1fb0 0x7fbfb80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 -- 172.21.15.83:0/964828600 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb80b1fb0 msgr2=0x7fbfb80cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb80b1fb0 0x7fbfb80cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 -- 172.21.15.83:0/964828600 >> v1:172.21.15.90:6789/0 conn(0x7fbfb815d0e0 legacy=0x7fbfb80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 -- 172.21.15.83:0/964828600 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80cb4e0 con 0x7fbfb815cd10 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbd562700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb80b1fb0 0x7fbfb80cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfbcd61700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb815cd10 0x7fbfb80ca970 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0098f0 tx=0x7fbfac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24b6aa2a161c53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfb77fe700 1 -- 172.21.15.83:0/964828600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00b8a0 con 0x7fbfb815cd10 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfb77fe700 1 -- 172.21.15.83:0/964828600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00ba00 con 0x7fbfb815cd10 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.259+0000 7fbfb77fe700 1 -- 172.21.15.83:0/964828600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00bd90 con 0x7fbfb815cd10 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 -- 172.21.15.83:0/964828600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb815cd10 msgr2=0x7fbfb80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb815cd10 0x7fbfb80ca970 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0098f0 tx=0x7fbfac009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 -- 172.21.15.83:0/964828600 shutdown_connections 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb80b1fb0 0x7fbfb80cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/964828600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb815cd10 0x7fbfb80ca970 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 -- 172.21.15.83:0/964828600 >> 172.21.15.83:0/964828600 conn(0x7fbfb806e9a0 msgr2=0x7fbfb80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 -- 172.21.15.83:0/964828600 shutdown_connections 2023-11-28T11:09:50.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.260+0000 7fbfbefc5700 1 -- 172.21.15.83:0/964828600 wait complete. 2023-11-28T11:09:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.261+0000 7fbfbefc5700 1 Processor -- start 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.261+0000 7fbfbefc5700 1 -- start start 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbefc5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 0x7fbfb80b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbefc5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb815cd10 0x7fbfb81523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbefc5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfb815d0e0 0x7fbfb80c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbefc5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80b4250 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbefc5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80b5090 con 0x7fbfb815d0e0 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbefc5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8152920 con 0x7fbfb815cd10 2023-11-28T11:09:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 0x7fbfb80b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 0x7fbfb80b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50792/0 (socket says 172.21.15.83:50792) 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 -- 172.21.15.83:0/3028762653 learned_addr learned my addr 172.21.15.83:0/3028762653 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbd562700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfb815d0e0 0x7fbfb80c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfb7fff700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb815cd10 0x7fbfb81523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 -- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb815cd10 msgr2=0x7fbfb81523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb815cd10 0x7fbfb81523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 -- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfb815d0e0 msgr2=0x7fbfb80c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfb815d0e0 0x7fbfb80c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 -- 172.21.15.83:0/3028762653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac009550 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.262+0000 7fbfbcd61700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 0x7fbfb80b3d10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009c40 tx=0x7fbfac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.263+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004480 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.263+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac01f070 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.263+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb80cbcd0 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.263+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb80cb9f0 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.263+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0038c0 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.264+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb806b9b0 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.265+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbfac02a020 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.265+0000 7fbfb57fa700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf9c0682f0 0x7fbf9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.265+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbfac0877c0 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.265+0000 7fbfb7fff700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf9c0682f0 0x7fbf9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.265+0000 7fbfb7fff700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf9c0682f0 0x7fbf9c06a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80abdd0 tx=0x7fbfa4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.269+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbfac053f80 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.451+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfb80b1290 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.452+0000 7fbfb57fa700 1 -- 172.21.15.83:0/3028762653 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fbfac0575a0 con 0x7fbfb80b1fb0 2023-11-28T11:09:50.454 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:50.454 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:47.730175+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf9c0682f0 msgr2=0x7fbf9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf9c0682f0 0x7fbf9c06a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80abdd0 tx=0x7fbfa4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 msgr2=0x7fbfb80b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 0x7fbfb80b3d10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009c40 tx=0x7fbfac004750 comp rx=0 tx=0).stop 2023-11-28T11:09:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 shutdown_connections 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfb815d0e0 0x7fbfb80c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf9c0682f0 0x7fbf9c06a7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb80b1fb0 0x7fbfb80b3d10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 --2- 172.21.15.83:0/3028762653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfb815cd10 0x7fbfb81523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 >> 172.21.15.83:0/3028762653 conn(0x7fbfb806e9a0 msgr2=0x7fbfb80cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 shutdown_connections 2023-11-28T11:09:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.454+0000 7fbfbefc5700 1 -- 172.21.15.83:0/3028762653 wait complete. 2023-11-28T11:09:50.468 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:09:50.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.586+0000 7f61cb501700 1 Processor -- start 2023-11-28T11:09:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.586+0000 7f61cb501700 1 -- start start 2023-11-28T11:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.587+0000 7f61cb501700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c406a950 0x7f61c415ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.587+0000 7f61cb501700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c415b160 con 0x7f61c40b5290 2023-11-28T11:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.587+0000 7f61cb501700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c415b440 con 0x7f61c40b4e40 2023-11-28T11:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.587+0000 7f61cb501700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415b720 con 0x7f61c406a950 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.587+0000 7f61c9a9e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f61c40b5290 0x7f61c4071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48664/0 (socket says 172.21.15.83:48664) 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.587+0000 7f61c9a9e700 1 -- 172.21.15.83:0/880304082 learned_addr learned my addr 172.21.15.83:0/880304082 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61c929d700 1 --2- 172.21.15.83:0/880304082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c406a950 0x7f61c415ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623810595 0 0) 0x7f61c415b160 con 0x7f61c40b5290 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b00035b0 con 0x7f61c40b5290 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625838415 0 0) 0x7f61c415b440 con 0x7f61c40b4e40 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c415b160 con 0x7f61c40b4e40 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2211872827 0 0) 0x7f61b00035b0 con 0x7f61c40b5290 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61c929d700 1 -- 172.21.15.83:0/880304082 >> v1:172.21.15.90:6789/0 conn(0x7f61c40b4e40 legacy=0x7f61c40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61c929d700 1 -- 172.21.15.83:0/880304082 >> v1:172.21.15.83:6789/0 conn(0x7f61c40b5290 legacy=0x7f61c4071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61c929d700 1 -- 172.21.15.83:0/880304082 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4071450 con 0x7f61c406a950 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61c929d700 1 --2- 172.21.15.83:0/880304082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c406a950 0x7f61c415ab30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009870 tx=0x7f61b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1771b98ef8d42a09 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0019f0 con 0x7f61c406a950 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4004030 con 0x7f61c406a950 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.588+0000 7f61bbfff700 1 -- 172.21.15.83:0/880304082 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40043a0 con 0x7f61c406a950 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 -- 172.21.15.83:0/880304082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c406a950 msgr2=0x7f61c415ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 --2- 172.21.15.83:0/880304082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c406a950 0x7f61c415ab30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009870 tx=0x7f61b4009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 -- 172.21.15.83:0/880304082 shutdown_connections 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 --2- 172.21.15.83:0/880304082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c406a950 0x7f61c415ab30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 -- 172.21.15.83:0/880304082 >> 172.21.15.83:0/880304082 conn(0x7f61c40cf4e0 msgr2=0x7f61c40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 -- 172.21.15.83:0/880304082 shutdown_connections 2023-11-28T11:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.589+0000 7f61cb501700 1 -- 172.21.15.83:0/880304082 wait complete. 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.590+0000 7f61cb501700 1 Processor -- start 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.590+0000 7f61cb501700 1 -- start start 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61cb501700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 0x7f61c4153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61cb501700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c40b4e40 0x7f61c40c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61cb501700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c40b5290 0x7f61c40b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61cb501700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4154450 con 0x7f61c406a950 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61cb501700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40b16b0 con 0x7f61c40b4e40 2023-11-28T11:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61cb501700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40cd610 con 0x7f61c40b5290 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 0x7f61c4153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 0x7f61c4153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50806/0 (socket says 172.21.15.83:50806) 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 -- 172.21.15.83:0/424244870 learned_addr learned my addr 172.21.15.83:0/424244870 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c9a9e700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c40b5290 0x7f61c40b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c8a9c700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c40b4e40 0x7f61c40c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 -- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c40b5290 msgr2=0x7f61c40b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c40b5290 0x7f61c40b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 -- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c40b4e40 msgr2=0x7f61c40c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c40b4e40 0x7f61c40c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 -- 172.21.15.83:0/424244870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40094a0 con 0x7f61c406a950 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61c929d700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 0x7f61c4153f10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f61b40097c0 tx=0x7f61b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003ad0 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.591+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b401f070 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.592+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003e90 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.592+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c40ca240 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.592+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c415ba00 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.593+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c406ad20 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.593+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f61b402a020 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.593+0000 7f61b9ffb700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61a80682f0 0x7f61a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.593+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f61b40871b0 con 0x7f61c406a950 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.593+0000 7f61c8a9c700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61a80682f0 0x7f61a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.594+0000 7f61c8a9c700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61a80682f0 0x7f61a806a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f61c406a830 tx=0x7f61c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.598+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61b40538d0 con 0x7f61c406a950 2023-11-28T11:09:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.776+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61c4074320 con 0x7f61a80682f0 2023-11-28T11:09:50.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.785+0000 7f61b9ffb700 1 -- 172.21.15.83:0/424244870 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311246 (secure 0 0 0) 0x7f61c4074320 con 0x7f61a80682f0 2023-11-28T11:09:50.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61a80682f0 msgr2=0x7f61a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61a80682f0 0x7f61a806a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f61c406a830 tx=0x7f61c0009040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 msgr2=0x7f61c4153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 0x7f61c4153f10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f61b40097c0 tx=0x7f61b4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 shutdown_connections 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f61c40b5290 0x7f61c40b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f61a80682f0 0x7f61a806a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c406a950 0x7f61c4153f10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 --2- 172.21.15.83:0/424244870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c40b4e40 0x7f61c40c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 >> 172.21.15.83:0/424244870 conn(0x7f61c40cf4e0 msgr2=0x7f61c4071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 shutdown_connections 2023-11-28T11:09:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.788+0000 7f61cb501700 1 -- 172.21.15.83:0/424244870 wait complete. 2023-11-28T11:09:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:09:50.801 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-11-28T11:09:49.583826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71324,"kb_used_data":4700,"kb_used_omap":0,"kb_used_meta":66560,"kb_avail":754903396,"statfs":{"total":773094113280,"available":773021077504,"internally_reserved":0,"allocated":4812800,"data_stored":2790595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68157440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.253262"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758150+0000","last_change":"2023-11-28T11:09:46.751690+0000","last_active":"2023-11-28T11:09:46.758150+0000","last_peered":"2023-11-28T11:09:46.758150+0000","last_clean":"2023-11-28T11:09:46.758150+0000","last_became_active":"2023-11-28T11:09:46.751356+0000","last_became_peered":"2023-11-28T11:09:46.751356+0000","last_unstale":"2023-11-28T11:09:46.758150+0000","last_undegraded":"2023-11-28T11:09:46.758150+0000","last_fullsized":"2023-11-28T11:09:46.758150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:08.097352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745831+0000","last_change":"2023-11-28T11:09:46.757613+0000","last_active":"2023-11-28T11:09:47.745831+0000","last_peered":"2023-11-28T11:09:47.745831+0000","last_clean":"2023-11-28T11:09:47.745831+0000","last_became_active":"2023-11-28T11:09:46.757330+0000","last_became_peered":"2023-11-28T11:09:46.757330+0000","last_unstale":"2023-11-28T11:09:47.745831+0000","last_undegraded":"2023-11-28T11:09:47.745831+0000","last_fullsized":"2023-11-28T11:09:47.745831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:20.150737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861947+0000","last_change":"2023-11-28T11:09:46.745960+0000","last_active":"2023-11-28T11:09:47.861947+0000","last_peered":"2023-11-28T11:09:47.861947+0000","last_clean":"2023-11-28T11:09:47.861947+0000","last_became_active":"2023-11-28T11:09:46.745636+0000","last_became_peered":"2023-11-28T11:09:46.745636+0000","last_unstale":"2023-11-28T11:09:47.861947+0000","last_undegraded":"2023-11-28T11:09:47.861947+0000","last_fullsized":"2023-11-28T11:09:47.861947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:29.539838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760001+0000","last_change":"2023-11-28T11:09:46.756943+0000","last_active":"2023-11-28T11:09:46.760001+0000","last_peered":"2023-11-28T11:09:46.760001+0000","last_clean":"2023-11-28T11:09:46.760001+0000","last_became_active":"2023-11-28T11:09:46.756443+0000","last_became_peered":"2023-11-28T11:09:46.756443+0000","last_unstale":"2023-11-28T11:09:46.760001+0000","last_undegraded":"2023-11-28T11:09:46.760001+0000","last_fullsized":"2023-11-28T11:09:46.760001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:44.827968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745690+0000","last_change":"2023-11-28T11:09:46.758214+0000","last_active":"2023-11-28T11:09:47.745690+0000","last_peered":"2023-11-28T11:09:47.745690+0000","last_clean":"2023-11-28T11:09:47.745690+0000","last_became_active":"2023-11-28T11:09:46.757946+0000","last_became_peered":"2023-11-28T11:09:46.757946+0000","last_unstale":"2023-11-28T11:09:47.745690+0000","last_undegraded":"2023-11-28T11:09:47.745690+0000","last_fullsized":"2023-11-28T11:09:47.745690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:18.425235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745491+0000","last_change":"2023-11-28T11:09:46.752141+0000","last_active":"2023-11-28T11:09:47.745491+0000","last_peered":"2023-11-28T11:09:47.745491+0000","last_clean":"2023-11-28T11:09:47.745491+0000","last_became_active":"2023-11-28T11:09:46.751875+0000","last_became_peered":"2023-11-28T11:09:46.751875+0000","last_unstale":"2023-11-28T11:09:47.745491+0000","last_undegraded":"2023-11-28T11:09:47.745491+0000","last_fullsized":"2023-11-28T11:09:47.745491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:51.026845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758402+0000","last_change":"2023-11-28T11:09:46.752527+0000","last_active":"2023-11-28T11:09:46.758402+0000","last_peered":"2023-11-28T11:09:46.758402+0000","last_clean":"2023-11-28T11:09:46.758402+0000","last_became_active":"2023-11-28T11:09:46.752105+0000","last_became_peered":"2023-11-28T11:09:46.752105+0000","last_unstale":"2023-11-28T11:09:46.758402+0000","last_undegraded":"2023-11-28T11:09:46.758402+0000","last_fullsized":"2023-11-28T11:09:46.758402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:19:13.804263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760004+0000","last_change":"2023-11-28T11:09:46.753183+0000","last_active":"2023-11-28T11:09:46.760004+0000","last_peered":"2023-11-28T11:09:46.760004+0000","last_clean":"2023-11-28T11:09:46.760004+0000","last_became_active":"2023-11-28T11:09:46.752773+0000","last_became_peered":"2023-11-28T11:09:46.752773+0000","last_unstale":"2023-11-28T11:09:46.760004+0000","last_undegraded":"2023-11-28T11:09:46.760004+0000","last_fullsized":"2023-11-28T11:09:46.760004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:44.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759009+0000","last_change":"2023-11-28T11:09:46.749118+0000","last_active":"2023-11-28T11:09:46.759009+0000","last_peered":"2023-11-28T11:09:46.759009+0000","last_clean":"2023-11-28T11:09:46.759009+0000","last_became_active":"2023-11-28T11:09:46.748473+0000","last_became_peered":"2023-11-28T11:09:46.748473+0000","last_unstale":"2023-11-28T11:09:46.759009+0000","last_undegraded":"2023-11-28T11:09:46.759009+0000","last_fullsized":"2023-11-28T11:09:46.759009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:48.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760042+0000","last_change":"2023-11-28T11:09:46.757114+0000","last_active":"2023-11-28T11:09:46.760042+0000","last_peered":"2023-11-28T11:09:46.760042+0000","last_clean":"2023-11-28T11:09:46.760042+0000","last_became_active":"2023-11-28T11:09:46.756807+0000","last_became_peered":"2023-11-28T11:09:46.756807+0000","last_unstale":"2023-11-28T11:09:46.760042+0000","last_undegraded":"2023-11-28T11:09:46.760042+0000","last_fullsized":"2023-11-28T11:09:46.760042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:14.881002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759600+0000","last_change":"2023-11-28T11:09:46.752851+0000","last_active":"2023-11-28T11:09:46.759600+0000","last_peered":"2023-11-28T11:09:46.759600+0000","last_clean":"2023-11-28T11:09:46.759600+0000","last_became_active":"2023-11-28T11:09:46.752639+0000","last_became_peered":"2023-11-28T11:09:46.752639+0000","last_unstale":"2023-11-28T11:09:46.759600+0000","last_undegraded":"2023-11-28T11:09:46.759600+0000","last_fullsized":"2023-11-28T11:09:46.759600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:51.026845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.746211+0000","last_change":"2023-11-28T11:09:46.755783+0000","last_active":"2023-11-28T11:09:47.746211+0000","last_peered":"2023-11-28T11:09:47.746211+0000","last_clean":"2023-11-28T11:09:47.746211+0000","last_became_active":"2023-11-28T11:09:46.755514+0000","last_became_peered":"2023-11-28T11:09:46.755514+0000","last_unstale":"2023-11-28T11:09:47.746211+0000","last_undegraded":"2023-11-28T11:09:47.746211+0000","last_fullsized":"2023-11-28T11:09:47.746211+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:49.226960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740514+0000","last_change":"2023-11-28T11:09:46.736680+0000","last_active":"2023-11-28T11:09:47.740514+0000","last_peered":"2023-11-28T11:09:47.740514+0000","last_clean":"2023-11-28T11:09:47.740514+0000","last_became_active":"2023-11-28T11:09:46.736429+0000","last_became_peered":"2023-11-28T11:09:46.736429+0000","last_unstale":"2023-11-28T11:09:47.740514+0000","last_undegraded":"2023-11-28T11:09:47.740514+0000","last_fullsized":"2023-11-28T11:09:47.740514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.589533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759217+0000","last_change":"2023-11-28T11:09:46.749970+0000","last_active":"2023-11-28T11:09:46.759217+0000","last_peered":"2023-11-28T11:09:46.759217+0000","last_clean":"2023-11-28T11:09:46.759217+0000","last_became_active":"2023-11-28T11:09:46.749689+0000","last_became_peered":"2023-11-28T11:09:46.749689+0000","last_unstale":"2023-11-28T11:09:46.759217+0000","last_undegraded":"2023-11-28T11:09:46.759217+0000","last_fullsized":"2023-11-28T11:09:46.759217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:16.235888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745427+0000","last_change":"2023-11-28T11:09:46.746813+0000","last_active":"2023-11-28T11:09:47.745427+0000","last_peered":"2023-11-28T11:09:47.745427+0000","last_clean":"2023-11-28T11:09:47.745427+0000","last_became_active":"2023-11-28T11:09:46.746569+0000","last_became_peered":"2023-11-28T11:09:46.746569+0000","last_unstale":"2023-11-28T11:09:47.745427+0000","last_undegraded":"2023-11-28T11:09:47.745427+0000","last_fullsized":"2023-11-28T11:09:47.745427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:27.120771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759591+0000","last_change":"2023-11-28T11:09:46.754933+0000","last_active":"2023-11-28T11:09:46.759591+0000","last_peered":"2023-11-28T11:09:46.759591+0000","last_clean":"2023-11-28T11:09:46.759591+0000","last_became_active":"2023-11-28T11:09:46.754285+0000","last_became_peered":"2023-11-28T11:09:46.754285+0000","last_unstale":"2023-11-28T11:09:46.759591+0000","last_undegraded":"2023-11-28T11:09:46.759591+0000","last_fullsized":"2023-11-28T11:09:46.759591+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:45.391857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862185+0000","last_change":"2023-11-28T11:09:46.750071+0000","last_active":"2023-11-28T11:09:47.862185+0000","last_peered":"2023-11-28T11:09:47.862185+0000","last_clean":"2023-11-28T11:09:47.862185+0000","last_became_active":"2023-11-28T11:09:46.749675+0000","last_became_peered":"2023-11-28T11:09:46.749675+0000","last_unstale":"2023-11-28T11:09:47.862185+0000","last_undegraded":"2023-11-28T11:09:47.862185+0000","last_fullsized":"2023-11-28T11:09:47.862185+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:16.235888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862097+0000","last_change":"2023-11-28T11:09:46.750304+0000","last_active":"2023-11-28T11:09:47.862097+0000","last_peered":"2023-11-28T11:09:47.862097+0000","last_clean":"2023-11-28T11:09:47.862097+0000","last_became_active":"2023-11-28T11:09:46.749782+0000","last_became_peered":"2023-11-28T11:09:46.749782+0000","last_unstale":"2023-11-28T11:09:47.862097+0000","last_undegraded":"2023-11-28T11:09:47.862097+0000","last_fullsized":"2023-11-28T11:09:47.862097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:27.120771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756443+0000","last_change":"2023-11-28T11:09:46.750823+0000","last_active":"2023-11-28T11:09:46.756443+0000","last_peered":"2023-11-28T11:09:46.756443+0000","last_clean":"2023-11-28T11:09:46.756443+0000","last_became_active":"2023-11-28T11:09:46.750555+0000","last_became_peered":"2023-11-28T11:09:46.750555+0000","last_unstale":"2023-11-28T11:09:46.756443+0000","last_undegraded":"2023-11-28T11:09:46.756443+0000","last_fullsized":"2023-11-28T11:09:46.756443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:42.348414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740641+0000","last_change":"2023-11-28T11:09:46.750041+0000","last_active":"2023-11-28T11:09:47.740641+0000","last_peered":"2023-11-28T11:09:47.740641+0000","last_clean":"2023-11-28T11:09:47.740641+0000","last_became_active":"2023-11-28T11:09:46.749777+0000","last_became_peered":"2023-11-28T11:09:46.749777+0000","last_unstale":"2023-11-28T11:09:47.740641+0000","last_undegraded":"2023-11-28T11:09:47.740641+0000","last_fullsized":"2023-11-28T11:09:47.740641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:49.590854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745588+0000","last_change":"2023-11-28T11:09:46.742426+0000","last_active":"2023-11-28T11:09:47.745588+0000","last_peered":"2023-11-28T11:09:47.745588+0000","last_clean":"2023-11-28T11:09:47.745588+0000","last_became_active":"2023-11-28T11:09:46.742179+0000","last_became_peered":"2023-11-28T11:09:46.742179+0000","last_unstale":"2023-11-28T11:09:47.745588+0000","last_undegraded":"2023-11-28T11:09:47.745588+0000","last_fullsized":"2023-11-28T11:09:47.745588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:14.881002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759989+0000","last_change":"2023-11-28T11:09:46.757275+0000","last_active":"2023-11-28T11:09:46.759989+0000","last_peered":"2023-11-28T11:09:46.759989+0000","last_clean":"2023-11-28T11:09:46.759989+0000","last_became_active":"2023-11-28T11:09:46.756588+0000","last_became_peered":"2023-11-28T11:09:46.756588+0000","last_unstale":"2023-11-28T11:09:46.759989+0000","last_undegraded":"2023-11-28T11:09:46.759989+0000","last_fullsized":"2023-11-28T11:09:46.759989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:15.299767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756421+0000","last_change":"2023-11-28T11:09:46.753037+0000","last_active":"2023-11-28T11:09:46.756421+0000","last_peered":"2023-11-28T11:09:46.756421+0000","last_clean":"2023-11-28T11:09:46.756421+0000","last_became_active":"2023-11-28T11:09:46.752659+0000","last_became_peered":"2023-11-28T11:09:46.752659+0000","last_unstale":"2023-11-28T11:09:46.756421+0000","last_undegraded":"2023-11-28T11:09:46.756421+0000","last_fullsized":"2023-11-28T11:09:46.756421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:46.451417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756368+0000","last_change":"2023-11-28T11:09:46.752148+0000","last_active":"2023-11-28T11:09:46.756368+0000","last_peered":"2023-11-28T11:09:46.756368+0000","last_clean":"2023-11-28T11:09:46.756368+0000","last_became_active":"2023-11-28T11:09:46.751824+0000","last_became_peered":"2023-11-28T11:09:46.751824+0000","last_unstale":"2023-11-28T11:09:46.756368+0000","last_undegraded":"2023-11-28T11:09:46.756368+0000","last_fullsized":"2023-11-28T11:09:46.756368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:31.765991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.755260+0000","last_change":"2023-11-28T11:09:46.745425+0000","last_active":"2023-11-28T11:09:46.755260+0000","last_peered":"2023-11-28T11:09:46.755260+0000","last_clean":"2023-11-28T11:09:46.755260+0000","last_became_active":"2023-11-28T11:09:46.745150+0000","last_became_peered":"2023-11-28T11:09:46.745150+0000","last_unstale":"2023-11-28T11:09:46.755260+0000","last_undegraded":"2023-11-28T11:09:46.755260+0000","last_fullsized":"2023-11-28T11:09:46.755260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:22.055097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740435+0000","last_change":"2023-11-28T11:09:46.736744+0000","last_active":"2023-11-28T11:09:47.740435+0000","last_peered":"2023-11-28T11:09:47.740435+0000","last_clean":"2023-11-28T11:09:47.740435+0000","last_became_active":"2023-11-28T11:09:46.736493+0000","last_became_peered":"2023-11-28T11:09:46.736493+0000","last_unstale":"2023-11-28T11:09:47.740435+0000","last_undegraded":"2023-11-28T11:09:47.740435+0000","last_fullsized":"2023-11-28T11:09:47.740435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:42.348414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861991+0000","last_change":"2023-11-28T11:09:46.747671+0000","last_active":"2023-11-28T11:09:47.861991+0000","last_peered":"2023-11-28T11:09:47.861991+0000","last_clean":"2023-11-28T11:09:47.861991+0000","last_became_active":"2023-11-28T11:09:46.747338+0000","last_became_peered":"2023-11-28T11:09:46.747338+0000","last_unstale":"2023-11-28T11:09:47.861991+0000","last_undegraded":"2023-11-28T11:09:47.861991+0000","last_fullsized":"2023-11-28T11:09:47.861991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:16.835323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745936+0000","last_change":"2023-11-28T11:09:46.756795+0000","last_active":"2023-11-28T11:09:47.745936+0000","last_peered":"2023-11-28T11:09:47.745936+0000","last_clean":"2023-11-28T11:09:47.745936+0000","last_became_active":"2023-11-28T11:09:46.756362+0000","last_became_peered":"2023-11-28T11:09:46.756362+0000","last_unstale":"2023-11-28T11:09:47.745936+0000","last_undegraded":"2023-11-28T11:09:47.745936+0000","last_fullsized":"2023-11-28T11:09:47.745936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:00.257162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.755682+0000","last_change":"2023-11-28T11:09:46.745597+0000","last_active":"2023-11-28T11:09:46.755682+0000","last_peered":"2023-11-28T11:09:46.755682+0000","last_clean":"2023-11-28T11:09:46.755682+0000","last_became_active":"2023-11-28T11:09:46.745001+0000","last_became_peered":"2023-11-28T11:09:46.745001+0000","last_unstale":"2023-11-28T11:09:46.755682+0000","last_undegraded":"2023-11-28T11:09:46.755682+0000","last_fullsized":"2023-11-28T11:09:46.755682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:43.908951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861888+0000","last_change":"2023-11-28T11:09:46.754613+0000","last_active":"2023-11-28T11:09:47.861888+0000","last_peered":"2023-11-28T11:09:47.861888+0000","last_clean":"2023-11-28T11:09:47.861888+0000","last_became_active":"2023-11-28T11:09:46.754346+0000","last_became_peered":"2023-11-28T11:09:46.754346+0000","last_unstale":"2023-11-28T11:09:47.861888+0000","last_undegraded":"2023-11-28T11:09:47.861888+0000","last_fullsized":"2023-11-28T11:09:47.861888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:15.299767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745437+0000","last_change":"2023-11-28T11:09:46.748324+0000","last_active":"2023-11-28T11:09:47.745437+0000","last_peered":"2023-11-28T11:09:47.745437+0000","last_clean":"2023-11-28T11:09:47.745437+0000","last_became_active":"2023-11-28T11:09:46.748064+0000","last_became_peered":"2023-11-28T11:09:46.748064+0000","last_unstale":"2023-11-28T11:09:47.745437+0000","last_undegraded":"2023-11-28T11:09:47.745437+0000","last_fullsized":"2023-11-28T11:09:47.745437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:48.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740381+0000","last_change":"2023-11-28T11:09:46.747634+0000","last_active":"2023-11-28T11:09:47.740381+0000","last_peered":"2023-11-28T11:09:47.740381+0000","last_clean":"2023-11-28T11:09:47.740381+0000","last_became_active":"2023-11-28T11:09:46.747363+0000","last_became_peered":"2023-11-28T11:09:46.747363+0000","last_unstale":"2023-11-28T11:09:47.740381+0000","last_undegraded":"2023-11-28T11:09:47.740381+0000","last_fullsized":"2023-11-28T11:09:47.740381+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:46.451417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886644+0000","last_change":"2023-11-28T11:09:46.754593+0000","last_active":"2023-11-28T11:09:47.886644+0000","last_peered":"2023-11-28T11:09:47.886644+0000","last_clean":"2023-11-28T11:09:47.886644+0000","last_became_active":"2023-11-28T11:09:46.754352+0000","last_became_peered":"2023-11-28T11:09:46.754352+0000","last_unstale":"2023-11-28T11:09:47.886644+0000","last_undegraded":"2023-11-28T11:09:47.886644+0000","last_fullsized":"2023-11-28T11:09:47.886644+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:49.590854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745841+0000","last_change":"2023-11-28T11:09:46.353696+0000","last_active":"2023-11-28T11:09:47.745841+0000","last_peered":"2023-11-28T11:09:47.745841+0000","last_clean":"2023-11-28T11:09:47.745841+0000","last_became_active":"2023-11-28T11:09:46.353359+0000","last_became_peered":"2023-11-28T11:09:46.353359+0000","last_unstale":"2023-11-28T11:09:47.745841+0000","last_undegraded":"2023-11-28T11:09:47.745841+0000","last_fullsized":"2023-11-28T11:09:47.745841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:59.255458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756283+0000","last_change":"2023-11-28T11:09:46.749596+0000","last_active":"2023-11-28T11:09:46.756283+0000","last_peered":"2023-11-28T11:09:46.756283+0000","last_clean":"2023-11-28T11:09:46.756283+0000","last_became_active":"2023-11-28T11:09:46.748982+0000","last_became_peered":"2023-11-28T11:09:46.748982+0000","last_unstale":"2023-11-28T11:09:46.756283+0000","last_undegraded":"2023-11-28T11:09:46.756283+0000","last_fullsized":"2023-11-28T11:09:46.756283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:49.590854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862170+0000","last_change":"2023-11-28T11:09:46.353786+0000","last_active":"2023-11-28T11:09:47.862170+0000","last_peered":"2023-11-28T11:09:47.862170+0000","last_clean":"2023-11-28T11:09:47.862170+0000","last_became_active":"2023-11-28T11:09:46.353520+0000","last_became_peered":"2023-11-28T11:09:46.353520+0000","last_unstale":"2023-11-28T11:09:47.862170+0000","last_undegraded":"2023-11-28T11:09:47.862170+0000","last_fullsized":"2023-11-28T11:09:47.862170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:46:50.326725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.746029+0000","last_change":"2023-11-28T11:09:46.756631+0000","last_active":"2023-11-28T11:09:47.746029+0000","last_peered":"2023-11-28T11:09:47.746029+0000","last_clean":"2023-11-28T11:09:47.746029+0000","last_became_active":"2023-11-28T11:09:46.756035+0000","last_became_peered":"2023-11-28T11:09:46.756035+0000","last_unstale":"2023-11-28T11:09:47.746029+0000","last_undegraded":"2023-11-28T11:09:47.746029+0000","last_fullsized":"2023-11-28T11:09:47.746029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:08.097352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886734+0000","last_change":"2023-11-28T11:09:46.343359+0000","last_active":"2023-11-28T11:09:47.886734+0000","last_peered":"2023-11-28T11:09:47.886734+0000","last_clean":"2023-11-28T11:09:47.886734+0000","last_became_active":"2023-11-28T11:09:46.343120+0000","last_became_peered":"2023-11-28T11:09:46.343120+0000","last_unstale":"2023-11-28T11:09:47.886734+0000","last_undegraded":"2023-11-28T11:09:47.886734+0000","last_fullsized":"2023-11-28T11:09:47.886734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:30.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886720+0000","last_change":"2023-11-28T11:09:46.744556+0000","last_active":"2023-11-28T11:09:47.886720+0000","last_peered":"2023-11-28T11:09:47.886720+0000","last_clean":"2023-11-28T11:09:47.886720+0000","last_became_active":"2023-11-28T11:09:46.744360+0000","last_became_peered":"2023-11-28T11:09:46.744360+0000","last_unstale":"2023-11-28T11:09:47.886720+0000","last_undegraded":"2023-11-28T11:09:47.886720+0000","last_fullsized":"2023-11-28T11:09:47.886720+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.589533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.757220+0000","last_change":"2023-11-28T11:09:46.345753+0000","last_active":"2023-11-28T11:09:46.757220+0000","last_peered":"2023-11-28T11:09:46.757220+0000","last_clean":"2023-11-28T11:09:46.757220+0000","last_became_active":"2023-11-28T11:09:46.345500+0000","last_became_peered":"2023-11-28T11:09:46.345500+0000","last_unstale":"2023-11-28T11:09:46.757220+0000","last_undegraded":"2023-11-28T11:09:46.757220+0000","last_fullsized":"2023-11-28T11:09:46.757220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:47.367935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861883+0000","last_change":"2023-11-28T11:09:46.747979+0000","last_active":"2023-11-28T11:09:47.861883+0000","last_peered":"2023-11-28T11:09:47.861883+0000","last_clean":"2023-11-28T11:09:47.861883+0000","last_became_active":"2023-11-28T11:09:46.747683+0000","last_became_peered":"2023-11-28T11:09:46.747683+0000","last_unstale":"2023-11-28T11:09:47.861883+0000","last_undegraded":"2023-11-28T11:09:47.861883+0000","last_fullsized":"2023-11-28T11:09:47.861883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:48.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745863+0000","last_change":"2023-11-28T11:09:46.348258+0000","last_active":"2023-11-28T11:09:47.745863+0000","last_peered":"2023-11-28T11:09:47.745863+0000","last_clean":"2023-11-28T11:09:47.745863+0000","last_became_active":"2023-11-28T11:09:46.347967+0000","last_became_peered":"2023-11-28T11:09:46.347967+0000","last_unstale":"2023-11-28T11:09:47.745863+0000","last_undegraded":"2023-11-28T11:09:47.745863+0000","last_fullsized":"2023-11-28T11:09:47.745863+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:54.133156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759660+0000","last_change":"2023-11-28T11:09:46.754849+0000","last_active":"2023-11-28T11:09:46.759660+0000","last_peered":"2023-11-28T11:09:46.759660+0000","last_clean":"2023-11-28T11:09:46.759660+0000","last_became_active":"2023-11-28T11:09:46.754589+0000","last_became_peered":"2023-11-28T11:09:46.754589+0000","last_unstale":"2023-11-28T11:09:46.759660+0000","last_undegraded":"2023-11-28T11:09:46.759660+0000","last_fullsized":"2023-11-28T11:09:46.759660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:52:53.068465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861967+0000","last_change":"2023-11-28T11:09:46.343574+0000","last_active":"2023-11-28T11:09:47.861967+0000","last_peered":"2023-11-28T11:09:47.861967+0000","last_clean":"2023-11-28T11:09:47.861967+0000","last_became_active":"2023-11-28T11:09:46.343287+0000","last_became_peered":"2023-11-28T11:09:46.343287+0000","last_unstale":"2023-11-28T11:09:47.861967+0000","last_undegraded":"2023-11-28T11:09:47.861967+0000","last_fullsized":"2023-11-28T11:09:47.861967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:17.423531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759974+0000","last_change":"2023-11-28T11:09:46.753025+0000","last_active":"2023-11-28T11:09:46.759974+0000","last_peered":"2023-11-28T11:09:46.759974+0000","last_clean":"2023-11-28T11:09:46.759974+0000","last_became_active":"2023-11-28T11:09:46.752427+0000","last_became_peered":"2023-11-28T11:09:46.752427+0000","last_unstale":"2023-11-28T11:09:46.759974+0000","last_undegraded":"2023-11-28T11:09:46.759974+0000","last_fullsized":"2023-11-28T11:09:46.759974+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:55:20.514437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745457+0000","last_change":"2023-11-28T11:09:46.341818+0000","last_active":"2023-11-28T11:09:47.745457+0000","last_peered":"2023-11-28T11:09:47.745457+0000","last_clean":"2023-11-28T11:09:47.745457+0000","last_became_active":"2023-11-28T11:09:46.341550+0000","last_became_peered":"2023-11-28T11:09:46.341550+0000","last_unstale":"2023-11-28T11:09:47.745457+0000","last_undegraded":"2023-11-28T11:09:47.745457+0000","last_fullsized":"2023-11-28T11:09:47.745457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:47.367935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756403+0000","last_change":"2023-11-28T11:09:46.752397+0000","last_active":"2023-11-28T11:09:46.756403+0000","last_peered":"2023-11-28T11:09:46.756403+0000","last_clean":"2023-11-28T11:09:46.756403+0000","last_became_active":"2023-11-28T11:09:46.752114+0000","last_became_peered":"2023-11-28T11:09:46.752114+0000","last_unstale":"2023-11-28T11:09:46.756403+0000","last_undegraded":"2023-11-28T11:09:46.756403+0000","last_fullsized":"2023-11-28T11:09:46.756403+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:18.176850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.743540+0000","last_change":"2023-11-28T11:09:46.347798+0000","last_active":"2023-11-28T11:09:46.743540+0000","last_peered":"2023-11-28T11:09:46.743540+0000","last_clean":"2023-11-28T11:09:46.743540+0000","last_became_active":"2023-11-28T11:09:46.347532+0000","last_became_peered":"2023-11-28T11:09:46.347532+0000","last_unstale":"2023-11-28T11:09:46.743540+0000","last_undegraded":"2023-11-28T11:09:46.743540+0000","last_fullsized":"2023-11-28T11:09:46.743540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:48.589150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862141+0000","last_change":"2023-11-28T11:09:46.750505+0000","last_active":"2023-11-28T11:09:47.862141+0000","last_peered":"2023-11-28T11:09:47.862141+0000","last_clean":"2023-11-28T11:09:47.862141+0000","last_became_active":"2023-11-28T11:09:46.750151+0000","last_became_peered":"2023-11-28T11:09:46.750151+0000","last_unstale":"2023-11-28T11:09:47.862141+0000","last_undegraded":"2023-11-28T11:09:47.862141+0000","last_fullsized":"2023-11-28T11:09:47.862141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:51.026845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862277+0000","last_change":"2023-11-28T11:09:46.345905+0000","last_active":"2023-11-28T11:09:47.862277+0000","last_peered":"2023-11-28T11:09:47.862277+0000","last_clean":"2023-11-28T11:09:47.862277+0000","last_became_active":"2023-11-28T11:09:46.345653+0000","last_became_peered":"2023-11-28T11:09:46.345653+0000","last_unstale":"2023-11-28T11:09:47.862277+0000","last_undegraded":"2023-11-28T11:09:47.862277+0000","last_fullsized":"2023-11-28T11:09:47.862277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:54.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758173+0000","last_change":"2023-11-28T11:09:46.752204+0000","last_active":"2023-11-28T11:09:46.758173+0000","last_peered":"2023-11-28T11:09:46.758173+0000","last_clean":"2023-11-28T11:09:46.758173+0000","last_became_active":"2023-11-28T11:09:46.751921+0000","last_became_peered":"2023-11-28T11:09:46.751921+0000","last_unstale":"2023-11-28T11:09:46.758173+0000","last_undegraded":"2023-11-28T11:09:46.758173+0000","last_fullsized":"2023-11-28T11:09:46.758173+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:00.257162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.745849+0000","last_change":"2023-11-28T11:09:46.350972+0000","last_active":"2023-11-28T11:09:46.745849+0000","last_peered":"2023-11-28T11:09:46.745849+0000","last_clean":"2023-11-28T11:09:46.745849+0000","last_became_active":"2023-11-28T11:09:46.350464+0000","last_became_peered":"2023-11-28T11:09:46.350464+0000","last_unstale":"2023-11-28T11:09:46.745849+0000","last_undegraded":"2023-11-28T11:09:46.745849+0000","last_fullsized":"2023-11-28T11:09:46.745849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:41.346710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745667+0000","last_change":"2023-11-28T11:09:46.736172+0000","last_active":"2023-11-28T11:09:47.745667+0000","last_peered":"2023-11-28T11:09:47.745667+0000","last_clean":"2023-11-28T11:09:47.745667+0000","last_became_active":"2023-11-28T11:09:46.735941+0000","last_became_peered":"2023-11-28T11:09:46.735941+0000","last_unstale":"2023-11-28T11:09:47.745667+0000","last_undegraded":"2023-11-28T11:09:47.745667+0000","last_fullsized":"2023-11-28T11:09:47.745667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:56.669254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756071+0000","last_change":"2023-11-28T11:09:46.354437+0000","last_active":"2023-11-28T11:09:46.756071+0000","last_peered":"2023-11-28T11:09:46.756071+0000","last_clean":"2023-11-28T11:09:46.756071+0000","last_became_active":"2023-11-28T11:09:46.354184+0000","last_became_peered":"2023-11-28T11:09:46.354184+0000","last_unstale":"2023-11-28T11:09:46.756071+0000","last_undegraded":"2023-11-28T11:09:46.756071+0000","last_fullsized":"2023-11-28T11:09:46.756071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758285+0000","last_change":"2023-11-28T11:09:46.753250+0000","last_active":"2023-11-28T11:09:46.758285+0000","last_peered":"2023-11-28T11:09:46.758285+0000","last_clean":"2023-11-28T11:09:46.758285+0000","last_became_active":"2023-11-28T11:09:46.752967+0000","last_became_peered":"2023-11-28T11:09:46.752967+0000","last_unstale":"2023-11-28T11:09:46.758285+0000","last_undegraded":"2023-11-28T11:09:46.758285+0000","last_fullsized":"2023-11-28T11:09:46.758285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:49.226960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740724+0000","last_change":"2023-11-28T11:09:46.350279+0000","last_active":"2023-11-28T11:09:47.740724+0000","last_peered":"2023-11-28T11:09:47.740724+0000","last_clean":"2023-11-28T11:09:47.740724+0000","last_became_active":"2023-11-28T11:09:46.349982+0000","last_became_peered":"2023-11-28T11:09:46.349982+0000","last_unstale":"2023-11-28T11:09:47.740724+0000","last_undegraded":"2023-11-28T11:09:47.740724+0000","last_fullsized":"2023-11-28T11:09:47.740724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:27.226965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759071+0000","last_change":"2023-11-28T11:09:46.755404+0000","last_active":"2023-11-28T11:09:46.759071+0000","last_peered":"2023-11-28T11:09:46.759071+0000","last_clean":"2023-11-28T11:09:46.759071+0000","last_became_active":"2023-11-28T11:09:46.755150+0000","last_became_peered":"2023-11-28T11:09:46.755150+0000","last_unstale":"2023-11-28T11:09:46.759071+0000","last_undegraded":"2023-11-28T11:09:46.759071+0000","last_fullsized":"2023-11-28T11:09:46.759071+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.589533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745681+0000","last_change":"2023-11-28T11:09:46.353705+0000","last_active":"2023-11-28T11:09:47.745681+0000","last_peered":"2023-11-28T11:09:47.745681+0000","last_clean":"2023-11-28T11:09:47.745681+0000","last_became_active":"2023-11-28T11:09:46.353381+0000","last_became_peered":"2023-11-28T11:09:46.353381+0000","last_unstale":"2023-11-28T11:09:47.745681+0000","last_undegraded":"2023-11-28T11:09:47.745681+0000","last_fullsized":"2023-11-28T11:09:47.745681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:17.423531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760028+0000","last_change":"2023-11-28T11:09:46.756821+0000","last_active":"2023-11-28T11:09:46.760028+0000","last_peered":"2023-11-28T11:09:46.760028+0000","last_clean":"2023-11-28T11:09:46.760028+0000","last_became_active":"2023-11-28T11:09:46.756391+0000","last_became_peered":"2023-11-28T11:09:46.756391+0000","last_unstale":"2023-11-28T11:09:46.760028+0000","last_undegraded":"2023-11-28T11:09:46.760028+0000","last_fullsized":"2023-11-28T11:09:46.760028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:48.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.741897+0000","last_change":"2023-11-28T11:09:46.344823+0000","last_active":"2023-11-28T11:09:46.741897+0000","last_peered":"2023-11-28T11:09:46.741897+0000","last_clean":"2023-11-28T11:09:46.741897+0000","last_became_active":"2023-11-28T11:09:46.344554+0000","last_became_peered":"2023-11-28T11:09:46.344554+0000","last_unstale":"2023-11-28T11:09:46.741897+0000","last_undegraded":"2023-11-28T11:09:46.741897+0000","last_fullsized":"2023-11-28T11:09:46.741897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:17.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745427+0000","last_change":"2023-11-28T11:09:46.738387+0000","last_active":"2023-11-28T11:09:47.745427+0000","last_peered":"2023-11-28T11:09:47.745427+0000","last_clean":"2023-11-28T11:09:47.745427+0000","last_became_active":"2023-11-28T11:09:46.738129+0000","last_became_peered":"2023-11-28T11:09:46.738129+0000","last_unstale":"2023-11-28T11:09:47.745427+0000","last_undegraded":"2023-11-28T11:09:47.745427+0000","last_fullsized":"2023-11-28T11:09:47.745427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:44.827968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745522+0000","last_change":"2023-11-28T11:09:46.354102+0000","last_active":"2023-11-28T11:09:47.745522+0000","last_peered":"2023-11-28T11:09:47.745522+0000","last_clean":"2023-11-28T11:09:47.745522+0000","last_became_active":"2023-11-28T11:09:46.353457+0000","last_became_peered":"2023-11-28T11:09:46.353457+0000","last_unstale":"2023-11-28T11:09:47.745522+0000","last_undegraded":"2023-11-28T11:09:47.745522+0000","last_fullsized":"2023-11-28T11:09:47.745522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:19.149033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758383+0000","last_change":"2023-11-28T11:09:46.752362+0000","last_active":"2023-11-28T11:09:46.758383+0000","last_peered":"2023-11-28T11:09:46.758383+0000","last_clean":"2023-11-28T11:09:46.758383+0000","last_became_active":"2023-11-28T11:09:46.751747+0000","last_became_peered":"2023-11-28T11:09:46.751747+0000","last_unstale":"2023-11-28T11:09:46.758383+0000","last_undegraded":"2023-11-28T11:09:46.758383+0000","last_fullsized":"2023-11-28T11:09:46.758383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:18.425235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760101+0000","last_change":"2023-11-28T11:09:46.348265+0000","last_active":"2023-11-28T11:09:46.760101+0000","last_peered":"2023-11-28T11:09:46.760101+0000","last_clean":"2023-11-28T11:09:46.760101+0000","last_became_active":"2023-11-28T11:09:46.348048+0000","last_became_peered":"2023-11-28T11:09:46.348048+0000","last_unstale":"2023-11-28T11:09:46.760101+0000","last_undegraded":"2023-11-28T11:09:46.760101+0000","last_fullsized":"2023-11-28T11:09:46.760101+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:05.587829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740494+0000","last_change":"2023-11-28T11:09:46.746949+0000","last_active":"2023-11-28T11:09:47.740494+0000","last_peered":"2023-11-28T11:09:47.740494+0000","last_clean":"2023-11-28T11:09:47.740494+0000","last_became_active":"2023-11-28T11:09:46.746701+0000","last_became_peered":"2023-11-28T11:09:46.746701+0000","last_unstale":"2023-11-28T11:09:47.740494+0000","last_undegraded":"2023-11-28T11:09:47.740494+0000","last_fullsized":"2023-11-28T11:09:47.740494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:34.088229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740564+0000","last_change":"2023-11-28T11:09:46.345912+0000","last_active":"2023-11-28T11:09:47.740564+0000","last_peered":"2023-11-28T11:09:47.740564+0000","last_clean":"2023-11-28T11:09:47.740564+0000","last_became_active":"2023-11-28T11:09:46.345614+0000","last_became_peered":"2023-11-28T11:09:46.345614+0000","last_unstale":"2023-11-28T11:09:47.740564+0000","last_undegraded":"2023-11-28T11:09:47.740564+0000","last_fullsized":"2023-11-28T11:09:47.740564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:10.014356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759681+0000","last_change":"2023-11-28T11:09:46.754975+0000","last_active":"2023-11-28T11:09:46.759681+0000","last_peered":"2023-11-28T11:09:46.759681+0000","last_clean":"2023-11-28T11:09:46.759681+0000","last_became_active":"2023-11-28T11:09:46.754741+0000","last_became_peered":"2023-11-28T11:09:46.754741+0000","last_unstale":"2023-11-28T11:09:46.759681+0000","last_undegraded":"2023-11-28T11:09:46.759681+0000","last_fullsized":"2023-11-28T11:09:46.759681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:16.835323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862231+0000","last_change":"2023-11-28T11:09:46.351503+0000","last_active":"2023-11-28T11:09:47.862231+0000","last_peered":"2023-11-28T11:09:47.862231+0000","last_clean":"2023-11-28T11:09:47.862231+0000","last_became_active":"2023-11-28T11:09:46.351204+0000","last_became_peered":"2023-11-28T11:09:46.351204+0000","last_unstale":"2023-11-28T11:09:47.862231+0000","last_undegraded":"2023-11-28T11:09:47.862231+0000","last_fullsized":"2023-11-28T11:09:47.862231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:07.095648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.744658+0000","last_change":"2023-11-28T11:09:46.741074+0000","last_active":"2023-11-28T11:09:46.744658+0000","last_peered":"2023-11-28T11:09:46.744658+0000","last_clean":"2023-11-28T11:09:46.744658+0000","last_became_active":"2023-11-28T11:09:46.740796+0000","last_became_peered":"2023-11-28T11:09:46.740796+0000","last_unstale":"2023-11-28T11:09:46.744658+0000","last_undegraded":"2023-11-28T11:09:46.744658+0000","last_fullsized":"2023-11-28T11:09:46.744658+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:42.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745779+0000","last_change":"2023-11-28T11:09:46.342518+0000","last_active":"2023-11-28T11:09:47.745779+0000","last_peered":"2023-11-28T11:09:47.745779+0000","last_clean":"2023-11-28T11:09:47.745779+0000","last_became_active":"2023-11-28T11:09:46.342243+0000","last_became_peered":"2023-11-28T11:09:46.342243+0000","last_unstale":"2023-11-28T11:09:47.745779+0000","last_undegraded":"2023-11-28T11:09:47.745779+0000","last_fullsized":"2023-11-28T11:09:47.745779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:33.086525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.755249+0000","last_change":"2023-11-28T11:09:46.745291+0000","last_active":"2023-11-28T11:09:46.755249+0000","last_peered":"2023-11-28T11:09:46.755249+0000","last_clean":"2023-11-28T11:09:46.755249+0000","last_became_active":"2023-11-28T11:09:46.744983+0000","last_became_peered":"2023-11-28T11:09:46.744983+0000","last_unstale":"2023-11-28T11:09:46.755249+0000","last_undegraded":"2023-11-28T11:09:46.755249+0000","last_fullsized":"2023-11-28T11:09:46.755249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:48.369639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745751+0000","last_change":"2023-11-28T11:09:46.354220+0000","last_active":"2023-11-28T11:09:47.745751+0000","last_peered":"2023-11-28T11:09:47.745751+0000","last_clean":"2023-11-28T11:09:47.745751+0000","last_became_active":"2023-11-28T11:09:46.353530+0000","last_became_peered":"2023-11-28T11:09:46.353530+0000","last_unstale":"2023-11-28T11:09:47.745751+0000","last_undegraded":"2023-11-28T11:09:47.745751+0000","last_fullsized":"2023-11-28T11:09:47.745751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:48.225256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745464+0000","last_change":"2023-11-28T11:09:46.738317+0000","last_active":"2023-11-28T11:09:47.745464+0000","last_peered":"2023-11-28T11:09:47.745464+0000","last_clean":"2023-11-28T11:09:47.745464+0000","last_became_active":"2023-11-28T11:09:46.738067+0000","last_became_peered":"2023-11-28T11:09:46.738067+0000","last_unstale":"2023-11-28T11:09:47.745464+0000","last_undegraded":"2023-11-28T11:09:47.745464+0000","last_fullsized":"2023-11-28T11:09:47.745464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:51:13.329980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745872+0000","last_change":"2023-11-28T11:09:46.348298+0000","last_active":"2023-11-28T11:09:47.745872+0000","last_peered":"2023-11-28T11:09:47.745872+0000","last_clean":"2023-11-28T11:09:47.745872+0000","last_became_active":"2023-11-28T11:09:46.348003+0000","last_became_peered":"2023-11-28T11:09:46.348003+0000","last_unstale":"2023-11-28T11:09:47.745872+0000","last_undegraded":"2023-11-28T11:09:47.745872+0000","last_fullsized":"2023-11-28T11:09:47.745872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:23.194273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886516+0000","last_change":"2023-11-28T11:09:46.749359+0000","last_active":"2023-11-28T11:09:47.886516+0000","last_peered":"2023-11-28T11:09:47.886516+0000","last_clean":"2023-11-28T11:09:47.886516+0000","last_became_active":"2023-11-28T11:09:46.749097+0000","last_became_peered":"2023-11-28T11:09:46.749097+0000","last_unstale":"2023-11-28T11:09:47.886516+0000","last_undegraded":"2023-11-28T11:09:47.886516+0000","last_fullsized":"2023-11-28T11:09:47.886516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:42.348414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745552+0000","last_change":"2023-11-28T11:09:46.353752+0000","last_active":"2023-11-28T11:09:47.745552+0000","last_peered":"2023-11-28T11:09:47.745552+0000","last_clean":"2023-11-28T11:09:47.745552+0000","last_became_active":"2023-11-28T11:09:46.353479+0000","last_became_peered":"2023-11-28T11:09:46.353479+0000","last_unstale":"2023-11-28T11:09:47.745552+0000","last_undegraded":"2023-11-28T11:09:47.745552+0000","last_fullsized":"2023-11-28T11:09:47.745552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740383+0000","last_change":"2023-11-28T11:09:46.752067+0000","last_active":"2023-11-28T11:09:47.740383+0000","last_peered":"2023-11-28T11:09:47.740383+0000","last_clean":"2023-11-28T11:09:47.740383+0000","last_became_active":"2023-11-28T11:09:46.751796+0000","last_became_peered":"2023-11-28T11:09:46.751796+0000","last_unstale":"2023-11-28T11:09:47.740383+0000","last_undegraded":"2023-11-28T11:09:47.740383+0000","last_fullsized":"2023-11-28T11:09:47.740383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:48.369639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.741870+0000","last_change":"2023-11-28T11:09:46.350586+0000","last_active":"2023-11-28T11:09:46.741870+0000","last_peered":"2023-11-28T11:09:46.741870+0000","last_clean":"2023-11-28T11:09:46.741870+0000","last_became_active":"2023-11-28T11:09:46.350251+0000","last_became_peered":"2023-11-28T11:09:46.350251+0000","last_unstale":"2023-11-28T11:09:46.741870+0000","last_undegraded":"2023-11-28T11:09:46.741870+0000","last_fullsized":"2023-11-28T11:09:46.741870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:33.086525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759500+0000","last_change":"2023-11-28T11:09:46.749118+0000","last_active":"2023-11-28T11:09:46.759500+0000","last_peered":"2023-11-28T11:09:46.759500+0000","last_clean":"2023-11-28T11:09:46.759500+0000","last_became_active":"2023-11-28T11:09:46.748629+0000","last_became_peered":"2023-11-28T11:09:46.748629+0000","last_unstale":"2023-11-28T11:09:46.759500+0000","last_undegraded":"2023-11-28T11:09:46.759500+0000","last_fullsized":"2023-11-28T11:09:46.759500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:27.120771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.741312+0000","last_change":"2023-11-28T11:09:46.350629+0000","last_active":"2023-11-28T11:09:46.741312+0000","last_peered":"2023-11-28T11:09:46.741312+0000","last_clean":"2023-11-28T11:09:46.741312+0000","last_became_active":"2023-11-28T11:09:46.350355+0000","last_became_peered":"2023-11-28T11:09:46.350355+0000","last_unstale":"2023-11-28T11:09:46.741312+0000","last_undegraded":"2023-11-28T11:09:46.741312+0000","last_fullsized":"2023-11-28T11:09:46.741312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:21.053393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745632+0000","last_change":"2023-11-28T11:09:46.741923+0000","last_active":"2023-11-28T11:09:47.745632+0000","last_peered":"2023-11-28T11:09:47.745632+0000","last_clean":"2023-11-28T11:09:47.745632+0000","last_became_active":"2023-11-28T11:09:46.741681+0000","last_became_peered":"2023-11-28T11:09:46.741681+0000","last_unstale":"2023-11-28T11:09:47.745632+0000","last_undegraded":"2023-11-28T11:09:47.745632+0000","last_fullsized":"2023-11-28T11:09:47.745632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.143857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862011+0000","last_change":"2023-11-28T11:09:46.350272+0000","last_active":"2023-11-28T11:09:47.862011+0000","last_peered":"2023-11-28T11:09:47.862011+0000","last_clean":"2023-11-28T11:09:47.862011+0000","last_became_active":"2023-11-28T11:09:46.350012+0000","last_became_peered":"2023-11-28T11:09:46.350012+0000","last_unstale":"2023-11-28T11:09:47.862011+0000","last_undegraded":"2023-11-28T11:09:47.862011+0000","last_fullsized":"2023-11-28T11:09:47.862011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:59.255458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.739263+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:09:46.739263+0000","last_peered":"2023-11-28T11:09:46.739263+0000","last_clean":"2023-11-28T11:09:46.739263+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:09:46.739263+0000","last_undegraded":"2023-11-28T11:09:46.739263+0000","last_fullsized":"2023-11-28T11:09:46.739263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745577+0000","last_change":"2023-11-28T11:09:46.745032+0000","last_active":"2023-11-28T11:09:47.745577+0000","last_peered":"2023-11-28T11:09:47.745577+0000","last_clean":"2023-11-28T11:09:47.745577+0000","last_became_active":"2023-11-28T11:09:46.744800+0000","last_became_peered":"2023-11-28T11:09:46.744800+0000","last_unstale":"2023-11-28T11:09:47.745577+0000","last_undegraded":"2023-11-28T11:09:47.745577+0000","last_fullsized":"2023-11-28T11:09:47.745577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:48.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.744951+0000","last_change":"2023-11-28T11:09:46.353748+0000","last_active":"2023-11-28T11:09:46.744951+0000","last_peered":"2023-11-28T11:09:46.744951+0000","last_clean":"2023-11-28T11:09:46.744951+0000","last_became_active":"2023-11-28T11:09:46.353474+0000","last_became_peered":"2023-11-28T11:09:46.353474+0000","last_unstale":"2023-11-28T11:09:46.744951+0000","last_undegraded":"2023-11-28T11:09:46.744951+0000","last_fullsized":"2023-11-28T11:09:46.744951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:30.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.735738+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:09:46.735738+0000","last_peered":"2023-11-28T11:09:46.735738+0000","last_clean":"2023-11-28T11:09:46.735738+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:09:46.735738+0000","last_undegraded":"2023-11-28T11:09:46.735738+0000","last_fullsized":"2023-11-28T11:09:46.735738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758310+0000","last_change":"2023-11-28T11:09:46.753729+0000","last_active":"2023-11-28T11:09:46.758310+0000","last_peered":"2023-11-28T11:09:46.758310+0000","last_clean":"2023-11-28T11:09:46.758310+0000","last_became_active":"2023-11-28T11:09:46.753458+0000","last_became_peered":"2023-11-28T11:09:46.753458+0000","last_unstale":"2023-11-28T11:09:46.758310+0000","last_undegraded":"2023-11-28T11:09:46.758310+0000","last_fullsized":"2023-11-28T11:09:46.758310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:55.924989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886532+0000","last_change":"2023-11-28T11:09:46.343344+0000","last_active":"2023-11-28T11:09:47.886532+0000","last_peered":"2023-11-28T11:09:47.886532+0000","last_clean":"2023-11-28T11:09:47.886532+0000","last_became_active":"2023-11-28T11:09:46.343094+0000","last_became_peered":"2023-11-28T11:09:46.343094+0000","last_unstale":"2023-11-28T11:09:47.886532+0000","last_undegraded":"2023-11-28T11:09:47.886532+0000","last_fullsized":"2023-11-28T11:09:47.886532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:17.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.740890+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:09:46.740890+0000","last_peered":"2023-11-28T11:09:46.740890+0000","last_clean":"2023-11-28T11:09:46.740890+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:09:46.740890+0000","last_undegraded":"2023-11-28T11:09:46.740890+0000","last_fullsized":"2023-11-28T11:09:46.740890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.340108+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:09:46.340108+0000","last_peered":"2023-11-28T11:09:46.340108+0000","last_clean":"2023-11-28T11:09:46.340108+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:09:46.340108+0000","last_undegraded":"2023-11-28T11:09:46.340108+0000","last_fullsized":"2023-11-28T11:09:46.340108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759981+0000","last_change":"2023-11-28T11:09:46.751203+0000","last_active":"2023-11-28T11:09:46.759981+0000","last_peered":"2023-11-28T11:09:46.759981+0000","last_clean":"2023-11-28T11:09:46.759981+0000","last_became_active":"2023-11-28T11:09:46.750985+0000","last_became_peered":"2023-11-28T11:09:46.750985+0000","last_unstale":"2023-11-28T11:09:46.759981+0000","last_undegraded":"2023-11-28T11:09:46.759981+0000","last_fullsized":"2023-11-28T11:09:46.759981+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:29.539838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.743081+0000","last_change":"2023-11-28T11:09:46.353882+0000","last_active":"2023-11-28T11:09:46.743081+0000","last_peered":"2023-11-28T11:09:46.743081+0000","last_clean":"2023-11-28T11:09:46.743081+0000","last_became_active":"2023-11-28T11:09:46.353630+0000","last_became_peered":"2023-11-28T11:09:46.353630+0000","last_unstale":"2023-11-28T11:09:46.743081+0000","last_undegraded":"2023-11-28T11:09:46.743081+0000","last_fullsized":"2023-11-28T11:09:46.743081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:42.907248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.339885+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:09:46.339885+0000","last_peered":"2023-11-28T11:09:46.339885+0000","last_clean":"2023-11-28T11:09:46.339885+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:09:46.339885+0000","last_undegraded":"2023-11-28T11:09:46.339885+0000","last_fullsized":"2023-11-28T11:09:46.339885+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760020+0000","last_change":"2023-11-28T11:09:46.756801+0000","last_active":"2023-11-28T11:09:46.760020+0000","last_peered":"2023-11-28T11:09:46.760020+0000","last_clean":"2023-11-28T11:09:46.760020+0000","last_became_active":"2023-11-28T11:09:46.756288+0000","last_became_peered":"2023-11-28T11:09:46.756288+0000","last_unstale":"2023-11-28T11:09:46.760020+0000","last_undegraded":"2023-11-28T11:09:46.760020+0000","last_fullsized":"2023-11-28T11:09:46.760020+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:51:13.329980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745891+0000","last_change":"2023-11-28T11:09:46.349099+0000","last_active":"2023-11-28T11:09:47.745891+0000","last_peered":"2023-11-28T11:09:47.745891+0000","last_clean":"2023-11-28T11:09:47.745891+0000","last_became_active":"2023-11-28T11:09:46.348751+0000","last_became_peered":"2023-11-28T11:09:46.348751+0000","last_unstale":"2023-11-28T11:09:47.745891+0000","last_undegraded":"2023-11-28T11:09:47.745891+0000","last_fullsized":"2023-11-28T11:09:47.745891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:42.907248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.340014+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:09:46.340014+0000","last_peered":"2023-11-28T11:09:46.340014+0000","last_clean":"2023-11-28T11:09:46.340014+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:09:46.340014+0000","last_undegraded":"2023-11-28T11:09:46.340014+0000","last_fullsized":"2023-11-28T11:09:46.340014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756302+0000","last_change":"2023-11-28T11:09:46.750779+0000","last_active":"2023-11-28T11:09:46.756302+0000","last_peered":"2023-11-28T11:09:46.756302+0000","last_clean":"2023-11-28T11:09:46.756302+0000","last_became_active":"2023-11-28T11:09:46.750474+0000","last_became_peered":"2023-11-28T11:09:46.750474+0000","last_unstale":"2023-11-28T11:09:46.756302+0000","last_undegraded":"2023-11-28T11:09:46.756302+0000","last_fullsized":"2023-11-28T11:09:46.756302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:34.088229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745780+0000","last_change":"2023-11-28T11:09:46.354038+0000","last_active":"2023-11-28T11:09:47.745780+0000","last_peered":"2023-11-28T11:09:47.745780+0000","last_clean":"2023-11-28T11:09:47.745780+0000","last_became_active":"2023-11-28T11:09:46.353778+0000","last_became_peered":"2023-11-28T11:09:46.353778+0000","last_unstale":"2023-11-28T11:09:47.745780+0000","last_undegraded":"2023-11-28T11:09:47.745780+0000","last_fullsized":"2023-11-28T11:09:47.745780+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:38.076123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.746503+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:09:46.746503+0000","last_peered":"2023-11-28T11:09:46.746503+0000","last_clean":"2023-11-28T11:09:46.746503+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:09:46.746503+0000","last_undegraded":"2023-11-28T11:09:46.746503+0000","last_fullsized":"2023-11-28T11:09:46.746503+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745469+0000","last_change":"2023-11-28T11:09:46.754787+0000","last_active":"2023-11-28T11:09:47.745469+0000","last_peered":"2023-11-28T11:09:47.745469+0000","last_clean":"2023-11-28T11:09:47.745469+0000","last_became_active":"2023-11-28T11:09:46.754537+0000","last_became_peered":"2023-11-28T11:09:46.754537+0000","last_unstale":"2023-11-28T11:09:47.745469+0000","last_undegraded":"2023-11-28T11:09:47.745469+0000","last_fullsized":"2023-11-28T11:09:47.745469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:55.924989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886472+0000","last_change":"2023-11-28T11:09:46.351186+0000","last_active":"2023-11-28T11:09:47.886472+0000","last_peered":"2023-11-28T11:09:47.886472+0000","last_clean":"2023-11-28T11:09:47.886472+0000","last_became_active":"2023-11-28T11:09:46.350863+0000","last_became_peered":"2023-11-28T11:09:46.350863+0000","last_unstale":"2023-11-28T11:09:47.886472+0000","last_undegraded":"2023-11-28T11:09:47.886472+0000","last_fullsized":"2023-11-28T11:09:47.886472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:44.390153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.347488+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:09:46.347488+0000","last_peered":"2023-11-28T11:09:46.347488+0000","last_clean":"2023-11-28T11:09:46.347488+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:09:46.347488+0000","last_undegraded":"2023-11-28T11:09:46.347488+0000","last_fullsized":"2023-11-28T11:09:46.347488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861859+0000","last_change":"2023-11-28T11:09:46.749073+0000","last_active":"2023-11-28T11:09:47.861859+0000","last_peered":"2023-11-28T11:09:47.861859+0000","last_clean":"2023-11-28T11:09:47.861859+0000","last_became_active":"2023-11-28T11:09:46.747878+0000","last_became_peered":"2023-11-28T11:09:46.747878+0000","last_unstale":"2023-11-28T11:09:47.861859+0000","last_undegraded":"2023-11-28T11:09:47.861859+0000","last_fullsized":"2023-11-28T11:09:47.861859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:48.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.745988+0000","last_change":"2023-11-28T11:09:46.350749+0000","last_active":"2023-11-28T11:09:46.745988+0000","last_peered":"2023-11-28T11:09:46.745988+0000","last_clean":"2023-11-28T11:09:46.745988+0000","last_became_active":"2023-11-28T11:09:46.350501+0000","last_became_peered":"2023-11-28T11:09:46.350501+0000","last_unstale":"2023-11-28T11:09:46.745988+0000","last_undegraded":"2023-11-28T11:09:46.745988+0000","last_fullsized":"2023-11-28T11:09:46.745988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:47.367935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.348273+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:09:46.348273+0000","last_peered":"2023-11-28T11:09:46.348273+0000","last_clean":"2023-11-28T11:09:46.348273+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:09:46.348273+0000","last_undegraded":"2023-11-28T11:09:46.348273+0000","last_fullsized":"2023-11-28T11:09:46.348273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9724,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362116,"statfs":{"total":96636764160,"available":96626806784,"internally_reserved":0,"allocated":1036288,"data_stored":787090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.374},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.623},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":442368,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.55500000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.56999999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.438}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.38},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.495}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9736,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362104,"statfs":{"total":96636764160,"available":96626794496,"internally_reserved":0,"allocated":1048576,"data_stored":794043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.253}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.317},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.371}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8692,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363148,"statfs":{"total":96636764160,"available":96627863552,"internally_reserved":0,"allocated":438272,"data_stored":195305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.224},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.311},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.6429999999999998,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.42899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8528,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363312,"statfs":{"total":96636764160,"available":96628031488,"internally_reserved":0,"allocated":466944,"data_stored":205175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.374}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.62}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":204227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.35899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.133,"5min":5.133,"15min":5.133},"last":0.376}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.52700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":205153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.623},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.64100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8644,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363196,"statfs":{"total":96636764160,"available":96627912704,"internally_reserved":0,"allocated":454656,"data_stored":203675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":8.4610000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":8.4830000000000005,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.601,"5min":6.601,"15min":6.601},"last":0.497}]},{"osd":2,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.71299999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.753},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.622},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:50.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.960+0000 7f6adf288700 1 Processor -- start 2023-11-28T11:09:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.960+0000 7f6adf288700 1 -- start start 2023-11-28T11:09:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6adf288700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6ad8159830 0x7f6ad80cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6adf288700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad806ca60 con 0x7f6ad806c5f0 2023-11-28T11:09:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6adf288700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad80cf790 con 0x7f6ad80b18a0 2023-11-28T11:09:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6adf288700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad815d480 con 0x7f6ad8159830 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6adc823700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ad806c5f0 0x7f6ad815de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48674/0 (socket says 172.21.15.83:48674) 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6adc823700 1 -- 172.21.15.83:0/3146072687 learned_addr learned my addr 172.21.15.83:0/3146072687 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6add024700 1 --2- 172.21.15.83:0/3146072687 >> v2:172.21.15.90:3300/0 conn(0x7f6ad8159830 0x7f6ad80cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761562082 0 0) 0x7f6ad806ca60 con 0x7f6ad806c5f0 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.961+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac40035b0 con 0x7f6ad806c5f0 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1052448746 0 0) 0x7f6ac40035b0 con 0x7f6ad806c5f0 2023-11-28T11:09:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 >> v2:172.21.15.90:3300/0 conn(0x7f6ad8159830 msgr2=0x7f6ad80cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 --2- 172.21.15.83:0/3146072687 >> v2:172.21.15.90:3300/0 conn(0x7f6ad8159830 0x7f6ad80cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 >> v1:172.21.15.90:6789/0 conn(0x7f6ad80b18a0 legacy=0x7f6ad80cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80cfa70 con 0x7f6ad806c5f0 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6ac8001670 con 0x7f6ad806c5f0 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 >> v1:172.21.15.83:6789/0 conn(0x7f6ad806c5f0 legacy=0x7f6ad815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ac4003a00 0x7f6ac4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ac4004850 0x7f6ac4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ac400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad815d480 con 0x7f6ac4005200 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac400c080 con 0x7f6ac4004850 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac400c240 con 0x7f6ac4003a00 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ac400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6adc823700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ac4004850 0x7f6ac4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add024700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ac4003a00 0x7f6ac4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 -- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ac4003a00 msgr2=0x7f6ac4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ac4003a00 0x7f6ac4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 -- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ac4004850 msgr2=0x7f6ac4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ac4004850 0x7f6ac4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 -- 172.21.15.83:0/3146072687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40035b0 con 0x7f6ac4005200 2023-11-28T11:09:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 -- 172.21.15.83:0/3146072687 reap_dead start 2023-11-28T11:09:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6add825700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ac400ba10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000c190 tx=0x7f6ac4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3253206715ce064 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0001db0 con 0x7f6ac4005200 2023-11-28T11:09:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad000ccf0 con 0x7f6ac4005200 2023-11-28T11:09:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.962+0000 7f6acffff700 1 -- 172.21.15.83:0/3146072687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0003c40 con 0x7f6ac4005200 2023-11-28T11:09:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 -- 172.21.15.83:0/3146072687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 msgr2=0x7f6ac400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ac400ba10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000c190 tx=0x7f6ac4005880 comp rx=0 tx=0).stop 2023-11-28T11:09:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 -- 172.21.15.83:0/3146072687 shutdown_connections 2023-11-28T11:09:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 --2- 172.21.15.83:0/3146072687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ac400ba10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 -- 172.21.15.83:0/3146072687 >> 172.21.15.83:0/3146072687 conn(0x7f6ad806e9a0 msgr2=0x7f6ad80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 -- 172.21.15.83:0/3146072687 shutdown_connections 2023-11-28T11:09:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.963+0000 7f6adf288700 1 -- 172.21.15.83:0/3146072687 wait complete. 2023-11-28T11:09:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.964+0000 7f6adf288700 1 Processor -- start 2023-11-28T11:09:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.964+0000 7f6adf288700 1 -- start start 2023-11-28T11:09:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adf288700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ad8154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adf288700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad80c5080 0x7f6ad814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adf288700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ad80c44d0 0x7f6ad8154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adf288700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad814e640 con 0x7f6ac4005200 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adf288700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80c57f0 con 0x7f6ad80c5080 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adf288700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80c5ad0 con 0x7f6ad80c44d0 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ad8154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ad8154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50826/0 (socket says 172.21.15.83:50826) 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 -- 172.21.15.83:0/2239423786 learned_addr learned my addr 172.21.15.83:0/2239423786 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adc823700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad80c5080 0x7f6ad814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add825700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ad80c44d0 0x7f6ad8154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 -- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ad80c44d0 msgr2=0x7f6ad8154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ad80c44d0 0x7f6ad8154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 -- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad80c5080 msgr2=0x7f6ad814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad80c5080 0x7f6ad814e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 -- 172.21.15.83:0/2239423786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad815d970 con 0x7f6ac4005200 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6adc823700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad80c5080 0x7f6ad814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.965+0000 7f6add024700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ad8154170 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40119a0 tx=0x7f6ad4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.966+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400f940 con 0x7f6ac4005200 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.966+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad400faa0 con 0x7f6ac4005200 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.966+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad815d690 con 0x7f6ac4005200 2023-11-28T11:09:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.966+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400fe30 con 0x7f6ac4005200 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.966+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad8159c00 con 0x7f6ac4005200 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.967+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad8074320 con 0x7f6ac4005200 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.967+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ad4014440 con 0x7f6ac4005200 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.967+0000 7f6acdffb700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6abc0682f0 0x7f6abc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.968+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6ad4086730 con 0x7f6ac4005200 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.968+0000 7f6adc823700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6abc0682f0 0x7f6abc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.968+0000 7f6adc823700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6abc0682f0 0x7f6abc06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ad806ae60 tx=0x7f6ac8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:50.972+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ad4052d00 con 0x7f6ac4005200 2023-11-28T11:09:51.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.152+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ad81558a0 con 0x7f6abc0682f0 2023-11-28T11:09:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.160+0000 7f6acdffb700 1 -- 172.21.15.83:0/2239423786 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311246 (secure 0 0 0) 0x7f6ad81558a0 con 0x7f6abc0682f0 2023-11-28T11:09:51.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6abc0682f0 msgr2=0x7f6abc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6abc0682f0 0x7f6abc06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ad806ae60 tx=0x7f6ac8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 msgr2=0x7f6ad8154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ad8154170 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40119a0 tx=0x7f6ad4011d60 comp rx=0 tx=0).stop 2023-11-28T11:09:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 shutdown_connections 2023-11-28T11:09:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6ad80c44d0 0x7f6ad8154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6abc0682f0 0x7f6abc06a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac4005200 0x7f6ad8154170 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 --2- 172.21.15.83:0/2239423786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad80c5080 0x7f6ad814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 >> 172.21.15.83:0/2239423786 conn(0x7f6ad806e9a0 msgr2=0x7f6ad80cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 shutdown_connections 2023-11-28T11:09:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.164+0000 7f6adf288700 1 -- 172.21.15.83:0/2239423786 wait complete. 2023-11-28T11:09:51.171 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:09:51.177 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-11-28T11:09:49.583826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71324,"kb_used_data":4700,"kb_used_omap":0,"kb_used_meta":66560,"kb_avail":754903396,"statfs":{"total":773094113280,"available":773021077504,"internally_reserved":0,"allocated":4812800,"data_stored":2790595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68157440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.253262"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758150+0000","last_change":"2023-11-28T11:09:46.751690+0000","last_active":"2023-11-28T11:09:46.758150+0000","last_peered":"2023-11-28T11:09:46.758150+0000","last_clean":"2023-11-28T11:09:46.758150+0000","last_became_active":"2023-11-28T11:09:46.751356+0000","last_became_peered":"2023-11-28T11:09:46.751356+0000","last_unstale":"2023-11-28T11:09:46.758150+0000","last_undegraded":"2023-11-28T11:09:46.758150+0000","last_fullsized":"2023-11-28T11:09:46.758150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:08.097352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745831+0000","last_change":"2023-11-28T11:09:46.757613+0000","last_active":"2023-11-28T11:09:47.745831+0000","last_peered":"2023-11-28T11:09:47.745831+0000","last_clean":"2023-11-28T11:09:47.745831+0000","last_became_active":"2023-11-28T11:09:46.757330+0000","last_became_peered":"2023-11-28T11:09:46.757330+0000","last_unstale":"2023-11-28T11:09:47.745831+0000","last_undegraded":"2023-11-28T11:09:47.745831+0000","last_fullsized":"2023-11-28T11:09:47.745831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:20.150737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861947+0000","last_change":"2023-11-28T11:09:46.745960+0000","last_active":"2023-11-28T11:09:47.861947+0000","last_peered":"2023-11-28T11:09:47.861947+0000","last_clean":"2023-11-28T11:09:47.861947+0000","last_became_active":"2023-11-28T11:09:46.745636+0000","last_became_peered":"2023-11-28T11:09:46.745636+0000","last_unstale":"2023-11-28T11:09:47.861947+0000","last_undegraded":"2023-11-28T11:09:47.861947+0000","last_fullsized":"2023-11-28T11:09:47.861947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:29.539838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760001+0000","last_change":"2023-11-28T11:09:46.756943+0000","last_active":"2023-11-28T11:09:46.760001+0000","last_peered":"2023-11-28T11:09:46.760001+0000","last_clean":"2023-11-28T11:09:46.760001+0000","last_became_active":"2023-11-28T11:09:46.756443+0000","last_became_peered":"2023-11-28T11:09:46.756443+0000","last_unstale":"2023-11-28T11:09:46.760001+0000","last_undegraded":"2023-11-28T11:09:46.760001+0000","last_fullsized":"2023-11-28T11:09:46.760001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:44.827968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745690+0000","last_change":"2023-11-28T11:09:46.758214+0000","last_active":"2023-11-28T11:09:47.745690+0000","last_peered":"2023-11-28T11:09:47.745690+0000","last_clean":"2023-11-28T11:09:47.745690+0000","last_became_active":"2023-11-28T11:09:46.757946+0000","last_became_peered":"2023-11-28T11:09:46.757946+0000","last_unstale":"2023-11-28T11:09:47.745690+0000","last_undegraded":"2023-11-28T11:09:47.745690+0000","last_fullsized":"2023-11-28T11:09:47.745690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:18.425235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745491+0000","last_change":"2023-11-28T11:09:46.752141+0000","last_active":"2023-11-28T11:09:47.745491+0000","last_peered":"2023-11-28T11:09:47.745491+0000","last_clean":"2023-11-28T11:09:47.745491+0000","last_became_active":"2023-11-28T11:09:46.751875+0000","last_became_peered":"2023-11-28T11:09:46.751875+0000","last_unstale":"2023-11-28T11:09:47.745491+0000","last_undegraded":"2023-11-28T11:09:47.745491+0000","last_fullsized":"2023-11-28T11:09:47.745491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:51.026845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758402+0000","last_change":"2023-11-28T11:09:46.752527+0000","last_active":"2023-11-28T11:09:46.758402+0000","last_peered":"2023-11-28T11:09:46.758402+0000","last_clean":"2023-11-28T11:09:46.758402+0000","last_became_active":"2023-11-28T11:09:46.752105+0000","last_became_peered":"2023-11-28T11:09:46.752105+0000","last_unstale":"2023-11-28T11:09:46.758402+0000","last_undegraded":"2023-11-28T11:09:46.758402+0000","last_fullsized":"2023-11-28T11:09:46.758402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:19:13.804263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760004+0000","last_change":"2023-11-28T11:09:46.753183+0000","last_active":"2023-11-28T11:09:46.760004+0000","last_peered":"2023-11-28T11:09:46.760004+0000","last_clean":"2023-11-28T11:09:46.760004+0000","last_became_active":"2023-11-28T11:09:46.752773+0000","last_became_peered":"2023-11-28T11:09:46.752773+0000","last_unstale":"2023-11-28T11:09:46.760004+0000","last_undegraded":"2023-11-28T11:09:46.760004+0000","last_fullsized":"2023-11-28T11:09:46.760004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:44.101865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759009+0000","last_change":"2023-11-28T11:09:46.749118+0000","last_active":"2023-11-28T11:09:46.759009+0000","last_peered":"2023-11-28T11:09:46.759009+0000","last_clean":"2023-11-28T11:09:46.759009+0000","last_became_active":"2023-11-28T11:09:46.748473+0000","last_became_peered":"2023-11-28T11:09:46.748473+0000","last_unstale":"2023-11-28T11:09:46.759009+0000","last_undegraded":"2023-11-28T11:09:46.759009+0000","last_fullsized":"2023-11-28T11:09:46.759009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:48.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760042+0000","last_change":"2023-11-28T11:09:46.757114+0000","last_active":"2023-11-28T11:09:46.760042+0000","last_peered":"2023-11-28T11:09:46.760042+0000","last_clean":"2023-11-28T11:09:46.760042+0000","last_became_active":"2023-11-28T11:09:46.756807+0000","last_became_peered":"2023-11-28T11:09:46.756807+0000","last_unstale":"2023-11-28T11:09:46.760042+0000","last_undegraded":"2023-11-28T11:09:46.760042+0000","last_fullsized":"2023-11-28T11:09:46.760042+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:14.881002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759600+0000","last_change":"2023-11-28T11:09:46.752851+0000","last_active":"2023-11-28T11:09:46.759600+0000","last_peered":"2023-11-28T11:09:46.759600+0000","last_clean":"2023-11-28T11:09:46.759600+0000","last_became_active":"2023-11-28T11:09:46.752639+0000","last_became_peered":"2023-11-28T11:09:46.752639+0000","last_unstale":"2023-11-28T11:09:46.759600+0000","last_undegraded":"2023-11-28T11:09:46.759600+0000","last_fullsized":"2023-11-28T11:09:46.759600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:51.026845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.746211+0000","last_change":"2023-11-28T11:09:46.755783+0000","last_active":"2023-11-28T11:09:47.746211+0000","last_peered":"2023-11-28T11:09:47.746211+0000","last_clean":"2023-11-28T11:09:47.746211+0000","last_became_active":"2023-11-28T11:09:46.755514+0000","last_became_peered":"2023-11-28T11:09:46.755514+0000","last_unstale":"2023-11-28T11:09:47.746211+0000","last_undegraded":"2023-11-28T11:09:47.746211+0000","last_fullsized":"2023-11-28T11:09:47.746211+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:49.226960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740514+0000","last_change":"2023-11-28T11:09:46.736680+0000","last_active":"2023-11-28T11:09:47.740514+0000","last_peered":"2023-11-28T11:09:47.740514+0000","last_clean":"2023-11-28T11:09:47.740514+0000","last_became_active":"2023-11-28T11:09:46.736429+0000","last_became_peered":"2023-11-28T11:09:46.736429+0000","last_unstale":"2023-11-28T11:09:47.740514+0000","last_undegraded":"2023-11-28T11:09:47.740514+0000","last_fullsized":"2023-11-28T11:09:47.740514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.589533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759217+0000","last_change":"2023-11-28T11:09:46.749970+0000","last_active":"2023-11-28T11:09:46.759217+0000","last_peered":"2023-11-28T11:09:46.759217+0000","last_clean":"2023-11-28T11:09:46.759217+0000","last_became_active":"2023-11-28T11:09:46.749689+0000","last_became_peered":"2023-11-28T11:09:46.749689+0000","last_unstale":"2023-11-28T11:09:46.759217+0000","last_undegraded":"2023-11-28T11:09:46.759217+0000","last_fullsized":"2023-11-28T11:09:46.759217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:16.235888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745427+0000","last_change":"2023-11-28T11:09:46.746813+0000","last_active":"2023-11-28T11:09:47.745427+0000","last_peered":"2023-11-28T11:09:47.745427+0000","last_clean":"2023-11-28T11:09:47.745427+0000","last_became_active":"2023-11-28T11:09:46.746569+0000","last_became_peered":"2023-11-28T11:09:46.746569+0000","last_unstale":"2023-11-28T11:09:47.745427+0000","last_undegraded":"2023-11-28T11:09:47.745427+0000","last_fullsized":"2023-11-28T11:09:47.745427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:27.120771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759591+0000","last_change":"2023-11-28T11:09:46.754933+0000","last_active":"2023-11-28T11:09:46.759591+0000","last_peered":"2023-11-28T11:09:46.759591+0000","last_clean":"2023-11-28T11:09:46.759591+0000","last_became_active":"2023-11-28T11:09:46.754285+0000","last_became_peered":"2023-11-28T11:09:46.754285+0000","last_unstale":"2023-11-28T11:09:46.759591+0000","last_undegraded":"2023-11-28T11:09:46.759591+0000","last_fullsized":"2023-11-28T11:09:46.759591+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:45.391857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862185+0000","last_change":"2023-11-28T11:09:46.750071+0000","last_active":"2023-11-28T11:09:47.862185+0000","last_peered":"2023-11-28T11:09:47.862185+0000","last_clean":"2023-11-28T11:09:47.862185+0000","last_became_active":"2023-11-28T11:09:46.749675+0000","last_became_peered":"2023-11-28T11:09:46.749675+0000","last_unstale":"2023-11-28T11:09:47.862185+0000","last_undegraded":"2023-11-28T11:09:47.862185+0000","last_fullsized":"2023-11-28T11:09:47.862185+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:16.235888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862097+0000","last_change":"2023-11-28T11:09:46.750304+0000","last_active":"2023-11-28T11:09:47.862097+0000","last_peered":"2023-11-28T11:09:47.862097+0000","last_clean":"2023-11-28T11:09:47.862097+0000","last_became_active":"2023-11-28T11:09:46.749782+0000","last_became_peered":"2023-11-28T11:09:46.749782+0000","last_unstale":"2023-11-28T11:09:47.862097+0000","last_undegraded":"2023-11-28T11:09:47.862097+0000","last_fullsized":"2023-11-28T11:09:47.862097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:27.120771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756443+0000","last_change":"2023-11-28T11:09:46.750823+0000","last_active":"2023-11-28T11:09:46.756443+0000","last_peered":"2023-11-28T11:09:46.756443+0000","last_clean":"2023-11-28T11:09:46.756443+0000","last_became_active":"2023-11-28T11:09:46.750555+0000","last_became_peered":"2023-11-28T11:09:46.750555+0000","last_unstale":"2023-11-28T11:09:46.756443+0000","last_undegraded":"2023-11-28T11:09:46.756443+0000","last_fullsized":"2023-11-28T11:09:46.756443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:42.348414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740641+0000","last_change":"2023-11-28T11:09:46.750041+0000","last_active":"2023-11-28T11:09:47.740641+0000","last_peered":"2023-11-28T11:09:47.740641+0000","last_clean":"2023-11-28T11:09:47.740641+0000","last_became_active":"2023-11-28T11:09:46.749777+0000","last_became_peered":"2023-11-28T11:09:46.749777+0000","last_unstale":"2023-11-28T11:09:47.740641+0000","last_undegraded":"2023-11-28T11:09:47.740641+0000","last_fullsized":"2023-11-28T11:09:47.740641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:49.590854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745588+0000","last_change":"2023-11-28T11:09:46.742426+0000","last_active":"2023-11-28T11:09:47.745588+0000","last_peered":"2023-11-28T11:09:47.745588+0000","last_clean":"2023-11-28T11:09:47.745588+0000","last_became_active":"2023-11-28T11:09:46.742179+0000","last_became_peered":"2023-11-28T11:09:46.742179+0000","last_unstale":"2023-11-28T11:09:47.745588+0000","last_undegraded":"2023-11-28T11:09:47.745588+0000","last_fullsized":"2023-11-28T11:09:47.745588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:14.881002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759989+0000","last_change":"2023-11-28T11:09:46.757275+0000","last_active":"2023-11-28T11:09:46.759989+0000","last_peered":"2023-11-28T11:09:46.759989+0000","last_clean":"2023-11-28T11:09:46.759989+0000","last_became_active":"2023-11-28T11:09:46.756588+0000","last_became_peered":"2023-11-28T11:09:46.756588+0000","last_unstale":"2023-11-28T11:09:46.759989+0000","last_undegraded":"2023-11-28T11:09:46.759989+0000","last_fullsized":"2023-11-28T11:09:46.759989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:15.299767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756421+0000","last_change":"2023-11-28T11:09:46.753037+0000","last_active":"2023-11-28T11:09:46.756421+0000","last_peered":"2023-11-28T11:09:46.756421+0000","last_clean":"2023-11-28T11:09:46.756421+0000","last_became_active":"2023-11-28T11:09:46.752659+0000","last_became_peered":"2023-11-28T11:09:46.752659+0000","last_unstale":"2023-11-28T11:09:46.756421+0000","last_undegraded":"2023-11-28T11:09:46.756421+0000","last_fullsized":"2023-11-28T11:09:46.756421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:46.451417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756368+0000","last_change":"2023-11-28T11:09:46.752148+0000","last_active":"2023-11-28T11:09:46.756368+0000","last_peered":"2023-11-28T11:09:46.756368+0000","last_clean":"2023-11-28T11:09:46.756368+0000","last_became_active":"2023-11-28T11:09:46.751824+0000","last_became_peered":"2023-11-28T11:09:46.751824+0000","last_unstale":"2023-11-28T11:09:46.756368+0000","last_undegraded":"2023-11-28T11:09:46.756368+0000","last_fullsized":"2023-11-28T11:09:46.756368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:31.765991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.755260+0000","last_change":"2023-11-28T11:09:46.745425+0000","last_active":"2023-11-28T11:09:46.755260+0000","last_peered":"2023-11-28T11:09:46.755260+0000","last_clean":"2023-11-28T11:09:46.755260+0000","last_became_active":"2023-11-28T11:09:46.745150+0000","last_became_peered":"2023-11-28T11:09:46.745150+0000","last_unstale":"2023-11-28T11:09:46.755260+0000","last_undegraded":"2023-11-28T11:09:46.755260+0000","last_fullsized":"2023-11-28T11:09:46.755260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:22.055097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740435+0000","last_change":"2023-11-28T11:09:46.736744+0000","last_active":"2023-11-28T11:09:47.740435+0000","last_peered":"2023-11-28T11:09:47.740435+0000","last_clean":"2023-11-28T11:09:47.740435+0000","last_became_active":"2023-11-28T11:09:46.736493+0000","last_became_peered":"2023-11-28T11:09:46.736493+0000","last_unstale":"2023-11-28T11:09:47.740435+0000","last_undegraded":"2023-11-28T11:09:47.740435+0000","last_fullsized":"2023-11-28T11:09:47.740435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:42.348414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861991+0000","last_change":"2023-11-28T11:09:46.747671+0000","last_active":"2023-11-28T11:09:47.861991+0000","last_peered":"2023-11-28T11:09:47.861991+0000","last_clean":"2023-11-28T11:09:47.861991+0000","last_became_active":"2023-11-28T11:09:46.747338+0000","last_became_peered":"2023-11-28T11:09:46.747338+0000","last_unstale":"2023-11-28T11:09:47.861991+0000","last_undegraded":"2023-11-28T11:09:47.861991+0000","last_fullsized":"2023-11-28T11:09:47.861991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:16.835323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745936+0000","last_change":"2023-11-28T11:09:46.756795+0000","last_active":"2023-11-28T11:09:47.745936+0000","last_peered":"2023-11-28T11:09:47.745936+0000","last_clean":"2023-11-28T11:09:47.745936+0000","last_became_active":"2023-11-28T11:09:46.756362+0000","last_became_peered":"2023-11-28T11:09:46.756362+0000","last_unstale":"2023-11-28T11:09:47.745936+0000","last_undegraded":"2023-11-28T11:09:47.745936+0000","last_fullsized":"2023-11-28T11:09:47.745936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:00.257162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.755682+0000","last_change":"2023-11-28T11:09:46.745597+0000","last_active":"2023-11-28T11:09:46.755682+0000","last_peered":"2023-11-28T11:09:46.755682+0000","last_clean":"2023-11-28T11:09:46.755682+0000","last_became_active":"2023-11-28T11:09:46.745001+0000","last_became_peered":"2023-11-28T11:09:46.745001+0000","last_unstale":"2023-11-28T11:09:46.755682+0000","last_undegraded":"2023-11-28T11:09:46.755682+0000","last_fullsized":"2023-11-28T11:09:46.755682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:43.908951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861888+0000","last_change":"2023-11-28T11:09:46.754613+0000","last_active":"2023-11-28T11:09:47.861888+0000","last_peered":"2023-11-28T11:09:47.861888+0000","last_clean":"2023-11-28T11:09:47.861888+0000","last_became_active":"2023-11-28T11:09:46.754346+0000","last_became_peered":"2023-11-28T11:09:46.754346+0000","last_unstale":"2023-11-28T11:09:47.861888+0000","last_undegraded":"2023-11-28T11:09:47.861888+0000","last_fullsized":"2023-11-28T11:09:47.861888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:15.299767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745437+0000","last_change":"2023-11-28T11:09:46.748324+0000","last_active":"2023-11-28T11:09:47.745437+0000","last_peered":"2023-11-28T11:09:47.745437+0000","last_clean":"2023-11-28T11:09:47.745437+0000","last_became_active":"2023-11-28T11:09:46.748064+0000","last_became_peered":"2023-11-28T11:09:46.748064+0000","last_unstale":"2023-11-28T11:09:47.745437+0000","last_undegraded":"2023-11-28T11:09:47.745437+0000","last_fullsized":"2023-11-28T11:09:47.745437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:48.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740381+0000","last_change":"2023-11-28T11:09:46.747634+0000","last_active":"2023-11-28T11:09:47.740381+0000","last_peered":"2023-11-28T11:09:47.740381+0000","last_clean":"2023-11-28T11:09:47.740381+0000","last_became_active":"2023-11-28T11:09:46.747363+0000","last_became_peered":"2023-11-28T11:09:46.747363+0000","last_unstale":"2023-11-28T11:09:47.740381+0000","last_undegraded":"2023-11-28T11:09:47.740381+0000","last_fullsized":"2023-11-28T11:09:47.740381+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:46.451417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886644+0000","last_change":"2023-11-28T11:09:46.754593+0000","last_active":"2023-11-28T11:09:47.886644+0000","last_peered":"2023-11-28T11:09:47.886644+0000","last_clean":"2023-11-28T11:09:47.886644+0000","last_became_active":"2023-11-28T11:09:46.754352+0000","last_became_peered":"2023-11-28T11:09:46.754352+0000","last_unstale":"2023-11-28T11:09:47.886644+0000","last_undegraded":"2023-11-28T11:09:47.886644+0000","last_fullsized":"2023-11-28T11:09:47.886644+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:49.590854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745841+0000","last_change":"2023-11-28T11:09:46.353696+0000","last_active":"2023-11-28T11:09:47.745841+0000","last_peered":"2023-11-28T11:09:47.745841+0000","last_clean":"2023-11-28T11:09:47.745841+0000","last_became_active":"2023-11-28T11:09:46.353359+0000","last_became_peered":"2023-11-28T11:09:46.353359+0000","last_unstale":"2023-11-28T11:09:47.745841+0000","last_undegraded":"2023-11-28T11:09:47.745841+0000","last_fullsized":"2023-11-28T11:09:47.745841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:59.255458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756283+0000","last_change":"2023-11-28T11:09:46.749596+0000","last_active":"2023-11-28T11:09:46.756283+0000","last_peered":"2023-11-28T11:09:46.756283+0000","last_clean":"2023-11-28T11:09:46.756283+0000","last_became_active":"2023-11-28T11:09:46.748982+0000","last_became_peered":"2023-11-28T11:09:46.748982+0000","last_unstale":"2023-11-28T11:09:46.756283+0000","last_undegraded":"2023-11-28T11:09:46.756283+0000","last_fullsized":"2023-11-28T11:09:46.756283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:49.590854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862170+0000","last_change":"2023-11-28T11:09:46.353786+0000","last_active":"2023-11-28T11:09:47.862170+0000","last_peered":"2023-11-28T11:09:47.862170+0000","last_clean":"2023-11-28T11:09:47.862170+0000","last_became_active":"2023-11-28T11:09:46.353520+0000","last_became_peered":"2023-11-28T11:09:46.353520+0000","last_unstale":"2023-11-28T11:09:47.862170+0000","last_undegraded":"2023-11-28T11:09:47.862170+0000","last_fullsized":"2023-11-28T11:09:47.862170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:46:50.326725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.746029+0000","last_change":"2023-11-28T11:09:46.756631+0000","last_active":"2023-11-28T11:09:47.746029+0000","last_peered":"2023-11-28T11:09:47.746029+0000","last_clean":"2023-11-28T11:09:47.746029+0000","last_became_active":"2023-11-28T11:09:46.756035+0000","last_became_peered":"2023-11-28T11:09:46.756035+0000","last_unstale":"2023-11-28T11:09:47.746029+0000","last_undegraded":"2023-11-28T11:09:47.746029+0000","last_fullsized":"2023-11-28T11:09:47.746029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:08.097352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886734+0000","last_change":"2023-11-28T11:09:46.343359+0000","last_active":"2023-11-28T11:09:47.886734+0000","last_peered":"2023-11-28T11:09:47.886734+0000","last_clean":"2023-11-28T11:09:47.886734+0000","last_became_active":"2023-11-28T11:09:46.343120+0000","last_became_peered":"2023-11-28T11:09:46.343120+0000","last_unstale":"2023-11-28T11:09:47.886734+0000","last_undegraded":"2023-11-28T11:09:47.886734+0000","last_fullsized":"2023-11-28T11:09:47.886734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:30.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886720+0000","last_change":"2023-11-28T11:09:46.744556+0000","last_active":"2023-11-28T11:09:47.886720+0000","last_peered":"2023-11-28T11:09:47.886720+0000","last_clean":"2023-11-28T11:09:47.886720+0000","last_became_active":"2023-11-28T11:09:46.744360+0000","last_became_peered":"2023-11-28T11:09:46.744360+0000","last_unstale":"2023-11-28T11:09:47.886720+0000","last_undegraded":"2023-11-28T11:09:47.886720+0000","last_fullsized":"2023-11-28T11:09:47.886720+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.589533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.757220+0000","last_change":"2023-11-28T11:09:46.345753+0000","last_active":"2023-11-28T11:09:46.757220+0000","last_peered":"2023-11-28T11:09:46.757220+0000","last_clean":"2023-11-28T11:09:46.757220+0000","last_became_active":"2023-11-28T11:09:46.345500+0000","last_became_peered":"2023-11-28T11:09:46.345500+0000","last_unstale":"2023-11-28T11:09:46.757220+0000","last_undegraded":"2023-11-28T11:09:46.757220+0000","last_fullsized":"2023-11-28T11:09:46.757220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:47.367935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861883+0000","last_change":"2023-11-28T11:09:46.747979+0000","last_active":"2023-11-28T11:09:47.861883+0000","last_peered":"2023-11-28T11:09:47.861883+0000","last_clean":"2023-11-28T11:09:47.861883+0000","last_became_active":"2023-11-28T11:09:46.747683+0000","last_became_peered":"2023-11-28T11:09:46.747683+0000","last_unstale":"2023-11-28T11:09:47.861883+0000","last_undegraded":"2023-11-28T11:09:47.861883+0000","last_fullsized":"2023-11-28T11:09:47.861883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:48.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745863+0000","last_change":"2023-11-28T11:09:46.348258+0000","last_active":"2023-11-28T11:09:47.745863+0000","last_peered":"2023-11-28T11:09:47.745863+0000","last_clean":"2023-11-28T11:09:47.745863+0000","last_became_active":"2023-11-28T11:09:46.347967+0000","last_became_peered":"2023-11-28T11:09:46.347967+0000","last_unstale":"2023-11-28T11:09:47.745863+0000","last_undegraded":"2023-11-28T11:09:47.745863+0000","last_fullsized":"2023-11-28T11:09:47.745863+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:54.133156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759660+0000","last_change":"2023-11-28T11:09:46.754849+0000","last_active":"2023-11-28T11:09:46.759660+0000","last_peered":"2023-11-28T11:09:46.759660+0000","last_clean":"2023-11-28T11:09:46.759660+0000","last_became_active":"2023-11-28T11:09:46.754589+0000","last_became_peered":"2023-11-28T11:09:46.754589+0000","last_unstale":"2023-11-28T11:09:46.759660+0000","last_undegraded":"2023-11-28T11:09:46.759660+0000","last_fullsized":"2023-11-28T11:09:46.759660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:52:53.068465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861967+0000","last_change":"2023-11-28T11:09:46.343574+0000","last_active":"2023-11-28T11:09:47.861967+0000","last_peered":"2023-11-28T11:09:47.861967+0000","last_clean":"2023-11-28T11:09:47.861967+0000","last_became_active":"2023-11-28T11:09:46.343287+0000","last_became_peered":"2023-11-28T11:09:46.343287+0000","last_unstale":"2023-11-28T11:09:47.861967+0000","last_undegraded":"2023-11-28T11:09:47.861967+0000","last_fullsized":"2023-11-28T11:09:47.861967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:17.423531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759974+0000","last_change":"2023-11-28T11:09:46.753025+0000","last_active":"2023-11-28T11:09:46.759974+0000","last_peered":"2023-11-28T11:09:46.759974+0000","last_clean":"2023-11-28T11:09:46.759974+0000","last_became_active":"2023-11-28T11:09:46.752427+0000","last_became_peered":"2023-11-28T11:09:46.752427+0000","last_unstale":"2023-11-28T11:09:46.759974+0000","last_undegraded":"2023-11-28T11:09:46.759974+0000","last_fullsized":"2023-11-28T11:09:46.759974+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:55:20.514437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745457+0000","last_change":"2023-11-28T11:09:46.341818+0000","last_active":"2023-11-28T11:09:47.745457+0000","last_peered":"2023-11-28T11:09:47.745457+0000","last_clean":"2023-11-28T11:09:47.745457+0000","last_became_active":"2023-11-28T11:09:46.341550+0000","last_became_peered":"2023-11-28T11:09:46.341550+0000","last_unstale":"2023-11-28T11:09:47.745457+0000","last_undegraded":"2023-11-28T11:09:47.745457+0000","last_fullsized":"2023-11-28T11:09:47.745457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:47.367935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756403+0000","last_change":"2023-11-28T11:09:46.752397+0000","last_active":"2023-11-28T11:09:46.756403+0000","last_peered":"2023-11-28T11:09:46.756403+0000","last_clean":"2023-11-28T11:09:46.756403+0000","last_became_active":"2023-11-28T11:09:46.752114+0000","last_became_peered":"2023-11-28T11:09:46.752114+0000","last_unstale":"2023-11-28T11:09:46.756403+0000","last_undegraded":"2023-11-28T11:09:46.756403+0000","last_fullsized":"2023-11-28T11:09:46.756403+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:18.176850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.743540+0000","last_change":"2023-11-28T11:09:46.347798+0000","last_active":"2023-11-28T11:09:46.743540+0000","last_peered":"2023-11-28T11:09:46.743540+0000","last_clean":"2023-11-28T11:09:46.743540+0000","last_became_active":"2023-11-28T11:09:46.347532+0000","last_became_peered":"2023-11-28T11:09:46.347532+0000","last_unstale":"2023-11-28T11:09:46.743540+0000","last_undegraded":"2023-11-28T11:09:46.743540+0000","last_fullsized":"2023-11-28T11:09:46.743540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:48.589150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862141+0000","last_change":"2023-11-28T11:09:46.750505+0000","last_active":"2023-11-28T11:09:47.862141+0000","last_peered":"2023-11-28T11:09:47.862141+0000","last_clean":"2023-11-28T11:09:47.862141+0000","last_became_active":"2023-11-28T11:09:46.750151+0000","last_became_peered":"2023-11-28T11:09:46.750151+0000","last_unstale":"2023-11-28T11:09:47.862141+0000","last_undegraded":"2023-11-28T11:09:47.862141+0000","last_fullsized":"2023-11-28T11:09:47.862141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:51.026845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862277+0000","last_change":"2023-11-28T11:09:46.345905+0000","last_active":"2023-11-28T11:09:47.862277+0000","last_peered":"2023-11-28T11:09:47.862277+0000","last_clean":"2023-11-28T11:09:47.862277+0000","last_became_active":"2023-11-28T11:09:46.345653+0000","last_became_peered":"2023-11-28T11:09:46.345653+0000","last_unstale":"2023-11-28T11:09:47.862277+0000","last_undegraded":"2023-11-28T11:09:47.862277+0000","last_fullsized":"2023-11-28T11:09:47.862277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:54.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758173+0000","last_change":"2023-11-28T11:09:46.752204+0000","last_active":"2023-11-28T11:09:46.758173+0000","last_peered":"2023-11-28T11:09:46.758173+0000","last_clean":"2023-11-28T11:09:46.758173+0000","last_became_active":"2023-11-28T11:09:46.751921+0000","last_became_peered":"2023-11-28T11:09:46.751921+0000","last_unstale":"2023-11-28T11:09:46.758173+0000","last_undegraded":"2023-11-28T11:09:46.758173+0000","last_fullsized":"2023-11-28T11:09:46.758173+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:00.257162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.745849+0000","last_change":"2023-11-28T11:09:46.350972+0000","last_active":"2023-11-28T11:09:46.745849+0000","last_peered":"2023-11-28T11:09:46.745849+0000","last_clean":"2023-11-28T11:09:46.745849+0000","last_became_active":"2023-11-28T11:09:46.350464+0000","last_became_peered":"2023-11-28T11:09:46.350464+0000","last_unstale":"2023-11-28T11:09:46.745849+0000","last_undegraded":"2023-11-28T11:09:46.745849+0000","last_fullsized":"2023-11-28T11:09:46.745849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:41.346710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745667+0000","last_change":"2023-11-28T11:09:46.736172+0000","last_active":"2023-11-28T11:09:47.745667+0000","last_peered":"2023-11-28T11:09:47.745667+0000","last_clean":"2023-11-28T11:09:47.745667+0000","last_became_active":"2023-11-28T11:09:46.735941+0000","last_became_peered":"2023-11-28T11:09:46.735941+0000","last_unstale":"2023-11-28T11:09:47.745667+0000","last_undegraded":"2023-11-28T11:09:47.745667+0000","last_fullsized":"2023-11-28T11:09:47.745667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:56.669254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756071+0000","last_change":"2023-11-28T11:09:46.354437+0000","last_active":"2023-11-28T11:09:46.756071+0000","last_peered":"2023-11-28T11:09:46.756071+0000","last_clean":"2023-11-28T11:09:46.756071+0000","last_became_active":"2023-11-28T11:09:46.354184+0000","last_became_peered":"2023-11-28T11:09:46.354184+0000","last_unstale":"2023-11-28T11:09:46.756071+0000","last_undegraded":"2023-11-28T11:09:46.756071+0000","last_fullsized":"2023-11-28T11:09:46.756071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758285+0000","last_change":"2023-11-28T11:09:46.753250+0000","last_active":"2023-11-28T11:09:46.758285+0000","last_peered":"2023-11-28T11:09:46.758285+0000","last_clean":"2023-11-28T11:09:46.758285+0000","last_became_active":"2023-11-28T11:09:46.752967+0000","last_became_peered":"2023-11-28T11:09:46.752967+0000","last_unstale":"2023-11-28T11:09:46.758285+0000","last_undegraded":"2023-11-28T11:09:46.758285+0000","last_fullsized":"2023-11-28T11:09:46.758285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:49.226960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740724+0000","last_change":"2023-11-28T11:09:46.350279+0000","last_active":"2023-11-28T11:09:47.740724+0000","last_peered":"2023-11-28T11:09:47.740724+0000","last_clean":"2023-11-28T11:09:47.740724+0000","last_became_active":"2023-11-28T11:09:46.349982+0000","last_became_peered":"2023-11-28T11:09:46.349982+0000","last_unstale":"2023-11-28T11:09:47.740724+0000","last_undegraded":"2023-11-28T11:09:47.740724+0000","last_fullsized":"2023-11-28T11:09:47.740724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:27.226965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759071+0000","last_change":"2023-11-28T11:09:46.755404+0000","last_active":"2023-11-28T11:09:46.759071+0000","last_peered":"2023-11-28T11:09:46.759071+0000","last_clean":"2023-11-28T11:09:46.759071+0000","last_became_active":"2023-11-28T11:09:46.755150+0000","last_became_peered":"2023-11-28T11:09:46.755150+0000","last_unstale":"2023-11-28T11:09:46.759071+0000","last_undegraded":"2023-11-28T11:09:46.759071+0000","last_fullsized":"2023-11-28T11:09:46.759071+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.589533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745681+0000","last_change":"2023-11-28T11:09:46.353705+0000","last_active":"2023-11-28T11:09:47.745681+0000","last_peered":"2023-11-28T11:09:47.745681+0000","last_clean":"2023-11-28T11:09:47.745681+0000","last_became_active":"2023-11-28T11:09:46.353381+0000","last_became_peered":"2023-11-28T11:09:46.353381+0000","last_unstale":"2023-11-28T11:09:47.745681+0000","last_undegraded":"2023-11-28T11:09:47.745681+0000","last_fullsized":"2023-11-28T11:09:47.745681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:17.423531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760028+0000","last_change":"2023-11-28T11:09:46.756821+0000","last_active":"2023-11-28T11:09:46.760028+0000","last_peered":"2023-11-28T11:09:46.760028+0000","last_clean":"2023-11-28T11:09:46.760028+0000","last_became_active":"2023-11-28T11:09:46.756391+0000","last_became_peered":"2023-11-28T11:09:46.756391+0000","last_unstale":"2023-11-28T11:09:46.760028+0000","last_undegraded":"2023-11-28T11:09:46.760028+0000","last_fullsized":"2023-11-28T11:09:46.760028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:48.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.741897+0000","last_change":"2023-11-28T11:09:46.344823+0000","last_active":"2023-11-28T11:09:46.741897+0000","last_peered":"2023-11-28T11:09:46.741897+0000","last_clean":"2023-11-28T11:09:46.741897+0000","last_became_active":"2023-11-28T11:09:46.344554+0000","last_became_peered":"2023-11-28T11:09:46.344554+0000","last_unstale":"2023-11-28T11:09:46.741897+0000","last_undegraded":"2023-11-28T11:09:46.741897+0000","last_fullsized":"2023-11-28T11:09:46.741897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:17.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745427+0000","last_change":"2023-11-28T11:09:46.738387+0000","last_active":"2023-11-28T11:09:47.745427+0000","last_peered":"2023-11-28T11:09:47.745427+0000","last_clean":"2023-11-28T11:09:47.745427+0000","last_became_active":"2023-11-28T11:09:46.738129+0000","last_became_peered":"2023-11-28T11:09:46.738129+0000","last_unstale":"2023-11-28T11:09:47.745427+0000","last_undegraded":"2023-11-28T11:09:47.745427+0000","last_fullsized":"2023-11-28T11:09:47.745427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:44.827968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745522+0000","last_change":"2023-11-28T11:09:46.354102+0000","last_active":"2023-11-28T11:09:47.745522+0000","last_peered":"2023-11-28T11:09:47.745522+0000","last_clean":"2023-11-28T11:09:47.745522+0000","last_became_active":"2023-11-28T11:09:46.353457+0000","last_became_peered":"2023-11-28T11:09:46.353457+0000","last_unstale":"2023-11-28T11:09:47.745522+0000","last_undegraded":"2023-11-28T11:09:47.745522+0000","last_fullsized":"2023-11-28T11:09:47.745522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:19.149033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758383+0000","last_change":"2023-11-28T11:09:46.752362+0000","last_active":"2023-11-28T11:09:46.758383+0000","last_peered":"2023-11-28T11:09:46.758383+0000","last_clean":"2023-11-28T11:09:46.758383+0000","last_became_active":"2023-11-28T11:09:46.751747+0000","last_became_peered":"2023-11-28T11:09:46.751747+0000","last_unstale":"2023-11-28T11:09:46.758383+0000","last_undegraded":"2023-11-28T11:09:46.758383+0000","last_fullsized":"2023-11-28T11:09:46.758383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:18.425235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760101+0000","last_change":"2023-11-28T11:09:46.348265+0000","last_active":"2023-11-28T11:09:46.760101+0000","last_peered":"2023-11-28T11:09:46.760101+0000","last_clean":"2023-11-28T11:09:46.760101+0000","last_became_active":"2023-11-28T11:09:46.348048+0000","last_became_peered":"2023-11-28T11:09:46.348048+0000","last_unstale":"2023-11-28T11:09:46.760101+0000","last_undegraded":"2023-11-28T11:09:46.760101+0000","last_fullsized":"2023-11-28T11:09:46.760101+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:05.587829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740494+0000","last_change":"2023-11-28T11:09:46.746949+0000","last_active":"2023-11-28T11:09:47.740494+0000","last_peered":"2023-11-28T11:09:47.740494+0000","last_clean":"2023-11-28T11:09:47.740494+0000","last_became_active":"2023-11-28T11:09:46.746701+0000","last_became_peered":"2023-11-28T11:09:46.746701+0000","last_unstale":"2023-11-28T11:09:47.740494+0000","last_undegraded":"2023-11-28T11:09:47.740494+0000","last_fullsized":"2023-11-28T11:09:47.740494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:34.088229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740564+0000","last_change":"2023-11-28T11:09:46.345912+0000","last_active":"2023-11-28T11:09:47.740564+0000","last_peered":"2023-11-28T11:09:47.740564+0000","last_clean":"2023-11-28T11:09:47.740564+0000","last_became_active":"2023-11-28T11:09:46.345614+0000","last_became_peered":"2023-11-28T11:09:46.345614+0000","last_unstale":"2023-11-28T11:09:47.740564+0000","last_undegraded":"2023-11-28T11:09:47.740564+0000","last_fullsized":"2023-11-28T11:09:47.740564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:10.014356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759681+0000","last_change":"2023-11-28T11:09:46.754975+0000","last_active":"2023-11-28T11:09:46.759681+0000","last_peered":"2023-11-28T11:09:46.759681+0000","last_clean":"2023-11-28T11:09:46.759681+0000","last_became_active":"2023-11-28T11:09:46.754741+0000","last_became_peered":"2023-11-28T11:09:46.754741+0000","last_unstale":"2023-11-28T11:09:46.759681+0000","last_undegraded":"2023-11-28T11:09:46.759681+0000","last_fullsized":"2023-11-28T11:09:46.759681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:16.835323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862231+0000","last_change":"2023-11-28T11:09:46.351503+0000","last_active":"2023-11-28T11:09:47.862231+0000","last_peered":"2023-11-28T11:09:47.862231+0000","last_clean":"2023-11-28T11:09:47.862231+0000","last_became_active":"2023-11-28T11:09:46.351204+0000","last_became_peered":"2023-11-28T11:09:46.351204+0000","last_unstale":"2023-11-28T11:09:47.862231+0000","last_undegraded":"2023-11-28T11:09:47.862231+0000","last_fullsized":"2023-11-28T11:09:47.862231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:07.095648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.744658+0000","last_change":"2023-11-28T11:09:46.741074+0000","last_active":"2023-11-28T11:09:46.744658+0000","last_peered":"2023-11-28T11:09:46.744658+0000","last_clean":"2023-11-28T11:09:46.744658+0000","last_became_active":"2023-11-28T11:09:46.740796+0000","last_became_peered":"2023-11-28T11:09:46.740796+0000","last_unstale":"2023-11-28T11:09:46.744658+0000","last_undegraded":"2023-11-28T11:09:46.744658+0000","last_fullsized":"2023-11-28T11:09:46.744658+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:42.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745779+0000","last_change":"2023-11-28T11:09:46.342518+0000","last_active":"2023-11-28T11:09:47.745779+0000","last_peered":"2023-11-28T11:09:47.745779+0000","last_clean":"2023-11-28T11:09:47.745779+0000","last_became_active":"2023-11-28T11:09:46.342243+0000","last_became_peered":"2023-11-28T11:09:46.342243+0000","last_unstale":"2023-11-28T11:09:47.745779+0000","last_undegraded":"2023-11-28T11:09:47.745779+0000","last_fullsized":"2023-11-28T11:09:47.745779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:33.086525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.755249+0000","last_change":"2023-11-28T11:09:46.745291+0000","last_active":"2023-11-28T11:09:46.755249+0000","last_peered":"2023-11-28T11:09:46.755249+0000","last_clean":"2023-11-28T11:09:46.755249+0000","last_became_active":"2023-11-28T11:09:46.744983+0000","last_became_peered":"2023-11-28T11:09:46.744983+0000","last_unstale":"2023-11-28T11:09:46.755249+0000","last_undegraded":"2023-11-28T11:09:46.755249+0000","last_fullsized":"2023-11-28T11:09:46.755249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:48.369639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745751+0000","last_change":"2023-11-28T11:09:46.354220+0000","last_active":"2023-11-28T11:09:47.745751+0000","last_peered":"2023-11-28T11:09:47.745751+0000","last_clean":"2023-11-28T11:09:47.745751+0000","last_became_active":"2023-11-28T11:09:46.353530+0000","last_became_peered":"2023-11-28T11:09:46.353530+0000","last_unstale":"2023-11-28T11:09:47.745751+0000","last_undegraded":"2023-11-28T11:09:47.745751+0000","last_fullsized":"2023-11-28T11:09:47.745751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:48.225256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745464+0000","last_change":"2023-11-28T11:09:46.738317+0000","last_active":"2023-11-28T11:09:47.745464+0000","last_peered":"2023-11-28T11:09:47.745464+0000","last_clean":"2023-11-28T11:09:47.745464+0000","last_became_active":"2023-11-28T11:09:46.738067+0000","last_became_peered":"2023-11-28T11:09:46.738067+0000","last_unstale":"2023-11-28T11:09:47.745464+0000","last_undegraded":"2023-11-28T11:09:47.745464+0000","last_fullsized":"2023-11-28T11:09:47.745464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:51:13.329980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745872+0000","last_change":"2023-11-28T11:09:46.348298+0000","last_active":"2023-11-28T11:09:47.745872+0000","last_peered":"2023-11-28T11:09:47.745872+0000","last_clean":"2023-11-28T11:09:47.745872+0000","last_became_active":"2023-11-28T11:09:46.348003+0000","last_became_peered":"2023-11-28T11:09:46.348003+0000","last_unstale":"2023-11-28T11:09:47.745872+0000","last_undegraded":"2023-11-28T11:09:47.745872+0000","last_fullsized":"2023-11-28T11:09:47.745872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:23.194273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886516+0000","last_change":"2023-11-28T11:09:46.749359+0000","last_active":"2023-11-28T11:09:47.886516+0000","last_peered":"2023-11-28T11:09:47.886516+0000","last_clean":"2023-11-28T11:09:47.886516+0000","last_became_active":"2023-11-28T11:09:46.749097+0000","last_became_peered":"2023-11-28T11:09:46.749097+0000","last_unstale":"2023-11-28T11:09:47.886516+0000","last_undegraded":"2023-11-28T11:09:47.886516+0000","last_fullsized":"2023-11-28T11:09:47.886516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:42.348414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745552+0000","last_change":"2023-11-28T11:09:46.353752+0000","last_active":"2023-11-28T11:09:47.745552+0000","last_peered":"2023-11-28T11:09:47.745552+0000","last_clean":"2023-11-28T11:09:47.745552+0000","last_became_active":"2023-11-28T11:09:46.353479+0000","last_became_peered":"2023-11-28T11:09:46.353479+0000","last_unstale":"2023-11-28T11:09:47.745552+0000","last_undegraded":"2023-11-28T11:09:47.745552+0000","last_fullsized":"2023-11-28T11:09:47.745552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.740383+0000","last_change":"2023-11-28T11:09:46.752067+0000","last_active":"2023-11-28T11:09:47.740383+0000","last_peered":"2023-11-28T11:09:47.740383+0000","last_clean":"2023-11-28T11:09:47.740383+0000","last_became_active":"2023-11-28T11:09:46.751796+0000","last_became_peered":"2023-11-28T11:09:46.751796+0000","last_unstale":"2023-11-28T11:09:47.740383+0000","last_undegraded":"2023-11-28T11:09:47.740383+0000","last_fullsized":"2023-11-28T11:09:47.740383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:48.369639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.741870+0000","last_change":"2023-11-28T11:09:46.350586+0000","last_active":"2023-11-28T11:09:46.741870+0000","last_peered":"2023-11-28T11:09:46.741870+0000","last_clean":"2023-11-28T11:09:46.741870+0000","last_became_active":"2023-11-28T11:09:46.350251+0000","last_became_peered":"2023-11-28T11:09:46.350251+0000","last_unstale":"2023-11-28T11:09:46.741870+0000","last_undegraded":"2023-11-28T11:09:46.741870+0000","last_fullsized":"2023-11-28T11:09:46.741870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:33.086525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759500+0000","last_change":"2023-11-28T11:09:46.749118+0000","last_active":"2023-11-28T11:09:46.759500+0000","last_peered":"2023-11-28T11:09:46.759500+0000","last_clean":"2023-11-28T11:09:46.759500+0000","last_became_active":"2023-11-28T11:09:46.748629+0000","last_became_peered":"2023-11-28T11:09:46.748629+0000","last_unstale":"2023-11-28T11:09:46.759500+0000","last_undegraded":"2023-11-28T11:09:46.759500+0000","last_fullsized":"2023-11-28T11:09:46.759500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:27.120771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.741312+0000","last_change":"2023-11-28T11:09:46.350629+0000","last_active":"2023-11-28T11:09:46.741312+0000","last_peered":"2023-11-28T11:09:46.741312+0000","last_clean":"2023-11-28T11:09:46.741312+0000","last_became_active":"2023-11-28T11:09:46.350355+0000","last_became_peered":"2023-11-28T11:09:46.350355+0000","last_unstale":"2023-11-28T11:09:46.741312+0000","last_undegraded":"2023-11-28T11:09:46.741312+0000","last_fullsized":"2023-11-28T11:09:46.741312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:21.053393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745632+0000","last_change":"2023-11-28T11:09:46.741923+0000","last_active":"2023-11-28T11:09:47.745632+0000","last_peered":"2023-11-28T11:09:47.745632+0000","last_clean":"2023-11-28T11:09:47.745632+0000","last_became_active":"2023-11-28T11:09:46.741681+0000","last_became_peered":"2023-11-28T11:09:46.741681+0000","last_unstale":"2023-11-28T11:09:47.745632+0000","last_undegraded":"2023-11-28T11:09:47.745632+0000","last_fullsized":"2023-11-28T11:09:47.745632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.143857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.862011+0000","last_change":"2023-11-28T11:09:46.350272+0000","last_active":"2023-11-28T11:09:47.862011+0000","last_peered":"2023-11-28T11:09:47.862011+0000","last_clean":"2023-11-28T11:09:47.862011+0000","last_became_active":"2023-11-28T11:09:46.350012+0000","last_became_peered":"2023-11-28T11:09:46.350012+0000","last_unstale":"2023-11-28T11:09:47.862011+0000","last_undegraded":"2023-11-28T11:09:47.862011+0000","last_fullsized":"2023-11-28T11:09:47.862011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:59.255458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.739263+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:09:46.739263+0000","last_peered":"2023-11-28T11:09:46.739263+0000","last_clean":"2023-11-28T11:09:46.739263+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:09:46.739263+0000","last_undegraded":"2023-11-28T11:09:46.739263+0000","last_fullsized":"2023-11-28T11:09:46.739263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745577+0000","last_change":"2023-11-28T11:09:46.745032+0000","last_active":"2023-11-28T11:09:47.745577+0000","last_peered":"2023-11-28T11:09:47.745577+0000","last_clean":"2023-11-28T11:09:47.745577+0000","last_became_active":"2023-11-28T11:09:46.744800+0000","last_became_peered":"2023-11-28T11:09:46.744800+0000","last_unstale":"2023-11-28T11:09:47.745577+0000","last_undegraded":"2023-11-28T11:09:47.745577+0000","last_fullsized":"2023-11-28T11:09:47.745577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:48.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.744951+0000","last_change":"2023-11-28T11:09:46.353748+0000","last_active":"2023-11-28T11:09:46.744951+0000","last_peered":"2023-11-28T11:09:46.744951+0000","last_clean":"2023-11-28T11:09:46.744951+0000","last_became_active":"2023-11-28T11:09:46.353474+0000","last_became_peered":"2023-11-28T11:09:46.353474+0000","last_unstale":"2023-11-28T11:09:46.744951+0000","last_undegraded":"2023-11-28T11:09:46.744951+0000","last_fullsized":"2023-11-28T11:09:46.744951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:30.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.735738+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:09:46.735738+0000","last_peered":"2023-11-28T11:09:46.735738+0000","last_clean":"2023-11-28T11:09:46.735738+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:09:46.735738+0000","last_undegraded":"2023-11-28T11:09:46.735738+0000","last_fullsized":"2023-11-28T11:09:46.735738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.758310+0000","last_change":"2023-11-28T11:09:46.753729+0000","last_active":"2023-11-28T11:09:46.758310+0000","last_peered":"2023-11-28T11:09:46.758310+0000","last_clean":"2023-11-28T11:09:46.758310+0000","last_became_active":"2023-11-28T11:09:46.753458+0000","last_became_peered":"2023-11-28T11:09:46.753458+0000","last_unstale":"2023-11-28T11:09:46.758310+0000","last_undegraded":"2023-11-28T11:09:46.758310+0000","last_fullsized":"2023-11-28T11:09:46.758310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:55.924989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886532+0000","last_change":"2023-11-28T11:09:46.343344+0000","last_active":"2023-11-28T11:09:47.886532+0000","last_peered":"2023-11-28T11:09:47.886532+0000","last_clean":"2023-11-28T11:09:47.886532+0000","last_became_active":"2023-11-28T11:09:46.343094+0000","last_became_peered":"2023-11-28T11:09:46.343094+0000","last_unstale":"2023-11-28T11:09:47.886532+0000","last_undegraded":"2023-11-28T11:09:47.886532+0000","last_fullsized":"2023-11-28T11:09:47.886532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:17.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.740890+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:09:46.740890+0000","last_peered":"2023-11-28T11:09:46.740890+0000","last_clean":"2023-11-28T11:09:46.740890+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:09:46.740890+0000","last_undegraded":"2023-11-28T11:09:46.740890+0000","last_fullsized":"2023-11-28T11:09:46.740890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.340108+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:09:46.340108+0000","last_peered":"2023-11-28T11:09:46.340108+0000","last_clean":"2023-11-28T11:09:46.340108+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:09:46.340108+0000","last_undegraded":"2023-11-28T11:09:46.340108+0000","last_fullsized":"2023-11-28T11:09:46.340108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.759981+0000","last_change":"2023-11-28T11:09:46.751203+0000","last_active":"2023-11-28T11:09:46.759981+0000","last_peered":"2023-11-28T11:09:46.759981+0000","last_clean":"2023-11-28T11:09:46.759981+0000","last_became_active":"2023-11-28T11:09:46.750985+0000","last_became_peered":"2023-11-28T11:09:46.750985+0000","last_unstale":"2023-11-28T11:09:46.759981+0000","last_undegraded":"2023-11-28T11:09:46.759981+0000","last_fullsized":"2023-11-28T11:09:46.759981+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:29.539838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.743081+0000","last_change":"2023-11-28T11:09:46.353882+0000","last_active":"2023-11-28T11:09:46.743081+0000","last_peered":"2023-11-28T11:09:46.743081+0000","last_clean":"2023-11-28T11:09:46.743081+0000","last_became_active":"2023-11-28T11:09:46.353630+0000","last_became_peered":"2023-11-28T11:09:46.353630+0000","last_unstale":"2023-11-28T11:09:46.743081+0000","last_undegraded":"2023-11-28T11:09:46.743081+0000","last_fullsized":"2023-11-28T11:09:46.743081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:42.907248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.339885+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:09:46.339885+0000","last_peered":"2023-11-28T11:09:46.339885+0000","last_clean":"2023-11-28T11:09:46.339885+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:09:46.339885+0000","last_undegraded":"2023-11-28T11:09:46.339885+0000","last_fullsized":"2023-11-28T11:09:46.339885+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.760020+0000","last_change":"2023-11-28T11:09:46.756801+0000","last_active":"2023-11-28T11:09:46.760020+0000","last_peered":"2023-11-28T11:09:46.760020+0000","last_clean":"2023-11-28T11:09:46.760020+0000","last_became_active":"2023-11-28T11:09:46.756288+0000","last_became_peered":"2023-11-28T11:09:46.756288+0000","last_unstale":"2023-11-28T11:09:46.760020+0000","last_undegraded":"2023-11-28T11:09:46.760020+0000","last_fullsized":"2023-11-28T11:09:46.760020+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:51:13.329980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745891+0000","last_change":"2023-11-28T11:09:46.349099+0000","last_active":"2023-11-28T11:09:47.745891+0000","last_peered":"2023-11-28T11:09:47.745891+0000","last_clean":"2023-11-28T11:09:47.745891+0000","last_became_active":"2023-11-28T11:09:46.348751+0000","last_became_peered":"2023-11-28T11:09:46.348751+0000","last_unstale":"2023-11-28T11:09:47.745891+0000","last_undegraded":"2023-11-28T11:09:47.745891+0000","last_fullsized":"2023-11-28T11:09:47.745891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:42.907248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.340014+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:09:46.340014+0000","last_peered":"2023-11-28T11:09:46.340014+0000","last_clean":"2023-11-28T11:09:46.340014+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:09:46.340014+0000","last_undegraded":"2023-11-28T11:09:46.340014+0000","last_fullsized":"2023-11-28T11:09:46.340014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.756302+0000","last_change":"2023-11-28T11:09:46.750779+0000","last_active":"2023-11-28T11:09:46.756302+0000","last_peered":"2023-11-28T11:09:46.756302+0000","last_clean":"2023-11-28T11:09:46.756302+0000","last_became_active":"2023-11-28T11:09:46.750474+0000","last_became_peered":"2023-11-28T11:09:46.750474+0000","last_unstale":"2023-11-28T11:09:46.756302+0000","last_undegraded":"2023-11-28T11:09:46.756302+0000","last_fullsized":"2023-11-28T11:09:46.756302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:34.088229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745780+0000","last_change":"2023-11-28T11:09:46.354038+0000","last_active":"2023-11-28T11:09:47.745780+0000","last_peered":"2023-11-28T11:09:47.745780+0000","last_clean":"2023-11-28T11:09:47.745780+0000","last_became_active":"2023-11-28T11:09:46.353778+0000","last_became_peered":"2023-11-28T11:09:46.353778+0000","last_unstale":"2023-11-28T11:09:47.745780+0000","last_undegraded":"2023-11-28T11:09:47.745780+0000","last_fullsized":"2023-11-28T11:09:47.745780+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:38.076123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.746503+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:09:46.746503+0000","last_peered":"2023-11-28T11:09:46.746503+0000","last_clean":"2023-11-28T11:09:46.746503+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:09:46.746503+0000","last_undegraded":"2023-11-28T11:09:46.746503+0000","last_fullsized":"2023-11-28T11:09:46.746503+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.745469+0000","last_change":"2023-11-28T11:09:46.754787+0000","last_active":"2023-11-28T11:09:47.745469+0000","last_peered":"2023-11-28T11:09:47.745469+0000","last_clean":"2023-11-28T11:09:47.745469+0000","last_became_active":"2023-11-28T11:09:46.754537+0000","last_became_peered":"2023-11-28T11:09:46.754537+0000","last_unstale":"2023-11-28T11:09:47.745469+0000","last_undegraded":"2023-11-28T11:09:47.745469+0000","last_fullsized":"2023-11-28T11:09:47.745469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:55.924989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.886472+0000","last_change":"2023-11-28T11:09:46.351186+0000","last_active":"2023-11-28T11:09:47.886472+0000","last_peered":"2023-11-28T11:09:47.886472+0000","last_clean":"2023-11-28T11:09:47.886472+0000","last_became_active":"2023-11-28T11:09:46.350863+0000","last_became_peered":"2023-11-28T11:09:46.350863+0000","last_unstale":"2023-11-28T11:09:47.886472+0000","last_undegraded":"2023-11-28T11:09:47.886472+0000","last_fullsized":"2023-11-28T11:09:47.886472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:44.390153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.347488+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:09:46.347488+0000","last_peered":"2023-11-28T11:09:46.347488+0000","last_clean":"2023-11-28T11:09:46.347488+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:09:46.347488+0000","last_undegraded":"2023-11-28T11:09:46.347488+0000","last_fullsized":"2023-11-28T11:09:46.347488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:09:47.861859+0000","last_change":"2023-11-28T11:09:46.749073+0000","last_active":"2023-11-28T11:09:47.861859+0000","last_peered":"2023-11-28T11:09:47.861859+0000","last_clean":"2023-11-28T11:09:47.861859+0000","last_became_active":"2023-11-28T11:09:46.747878+0000","last_became_peered":"2023-11-28T11:09:46.747878+0000","last_unstale":"2023-11-28T11:09:47.861859+0000","last_undegraded":"2023-11-28T11:09:47.861859+0000","last_fullsized":"2023-11-28T11:09:47.861859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:46.327999+0000","last_clean_scrub_stamp":"2023-11-28T11:09:46.327999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:48.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.745988+0000","last_change":"2023-11-28T11:09:46.350749+0000","last_active":"2023-11-28T11:09:46.745988+0000","last_peered":"2023-11-28T11:09:46.745988+0000","last_clean":"2023-11-28T11:09:46.745988+0000","last_became_active":"2023-11-28T11:09:46.350501+0000","last_became_peered":"2023-11-28T11:09:46.350501+0000","last_unstale":"2023-11-28T11:09:46.745988+0000","last_undegraded":"2023-11-28T11:09:46.745988+0000","last_fullsized":"2023-11-28T11:09:46.745988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:45.326295+0000","last_clean_scrub_stamp":"2023-11-28T11:09:45.326295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:47.367935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:46.348273+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:09:46.348273+0000","last_peered":"2023-11-28T11:09:46.348273+0000","last_clean":"2023-11-28T11:09:46.348273+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:09:46.348273+0000","last_undegraded":"2023-11-28T11:09:46.348273+0000","last_fullsized":"2023-11-28T11:09:46.348273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9724,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362116,"statfs":{"total":96636764160,"available":96626806784,"internally_reserved":0,"allocated":1036288,"data_stored":787090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.374},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.623},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":442368,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.55500000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.56999999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.438}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.38},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.495}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9736,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362104,"statfs":{"total":96636764160,"available":96626794496,"internally_reserved":0,"allocated":1048576,"data_stored":794043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.253}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.317},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.371}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8692,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363148,"statfs":{"total":96636764160,"available":96627863552,"internally_reserved":0,"allocated":438272,"data_stored":195305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.224},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.311},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.6429999999999998,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.42899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8528,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363312,"statfs":{"total":96636764160,"available":96628031488,"internally_reserved":0,"allocated":466944,"data_stored":205175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.374}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.62}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":204227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.35899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.133,"5min":5.133,"15min":5.133},"last":0.376}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.52700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":205153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.623},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.64100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8644,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363196,"statfs":{"total":96636764160,"available":96627912704,"internally_reserved":0,"allocated":454656,"data_stored":203675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":8.4610000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":8.4830000000000005,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.601,"5min":6.601,"15min":6.601},"last":0.497}]},{"osd":2,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.71299999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.753},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.622},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:51.183 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:09:51.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:09:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.343+0000 7f4552bd6700 1 Processor -- start 2023-11-28T11:09:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.343+0000 7f4552bd6700 1 -- start start 2023-11-28T11:09:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4552bd6700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f454c0cbb20 0x7f454c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4552bd6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f454c161230 0x7f454c160620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4552bd6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454c0cd380 con 0x7f454c0cbef0 2023-11-28T11:09:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4552bd6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0cc770 con 0x7f454c161230 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4552bd6700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0cca50 con 0x7f454c0cbb20 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f454bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f454c161230 0x7f454c160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4551173700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f454c0cbef0 0x7f454c0aea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48698/0 (socket says 172.21.15.83:48698) 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.344+0000 7f4551173700 1 -- 172.21.15.83:0/1902535966 learned_addr learned my addr 172.21.15.83:0/1902535966 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454bfff700 1 -- 172.21.15.83:0/1902535966 >> v2:172.21.15.90:3300/0 conn(0x7f454c0cbb20 msgr2=0x7f454c0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4550972700 1 --2- 172.21.15.83:0/1902535966 >> v2:172.21.15.90:3300/0 conn(0x7f454c0cbb20 0x7f454c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454bfff700 1 --2- 172.21.15.83:0/1902535966 >> v2:172.21.15.90:3300/0 conn(0x7f454c0cbb20 0x7f454c0b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454b7fe700 1 -- 172.21.15.83:0/1902535966 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856175301 0 0) 0x7f454c0cd380 con 0x7f454c0cbef0 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454bfff700 1 -- 172.21.15.83:0/1902535966 >> v1:172.21.15.83:6789/0 conn(0x7f454c0cbef0 legacy=0x7f454c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454bfff700 1 -- 172.21.15.83:0/1902535966 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0ccd30 con 0x7f454c161230 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454bfff700 1 --2- 172.21.15.83:0/1902535966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c161230 0x7f454c160620 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b0e20 tx=0x7f4538009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da923f4284a8c169 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454b7fe700 1 -- 172.21.15.83:0/1902535966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538011830 con 0x7f454c161230 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454b7fe700 1 -- 172.21.15.83:0/1902535966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538011990 con 0x7f454c161230 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f454b7fe700 1 -- 172.21.15.83:0/1902535966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538011d20 con 0x7f454c161230 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 -- 172.21.15.83:0/1902535966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c161230 msgr2=0x7f454c160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 --2- 172.21.15.83:0/1902535966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c161230 0x7f454c160620 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b0e20 tx=0x7f4538009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 -- 172.21.15.83:0/1902535966 shutdown_connections 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 --2- 172.21.15.83:0/1902535966 >> v2:172.21.15.90:3300/0 conn(0x7f454c0cbb20 0x7f454c0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 --2- 172.21.15.83:0/1902535966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c161230 0x7f454c160620 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 -- 172.21.15.83:0/1902535966 >> 172.21.15.83:0/1902535966 conn(0x7f454c0b2ca0 msgr2=0x7f454c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.345+0000 7f4552bd6700 1 -- 172.21.15.83:0/1902535966 shutdown_connections 2023-11-28T11:09:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.346+0000 7f4552bd6700 1 -- 172.21.15.83:0/1902535966 wait complete. 2023-11-28T11:09:51.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.347+0000 7f4552bd6700 1 Processor -- start 2023-11-28T11:09:51.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.347+0000 7f4552bd6700 1 -- start start 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.347+0000 7f4552bd6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c0cbb20 0x7f454c14d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.347+0000 7f4552bd6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 0x7f454c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4552bd6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f454c161230 0x7f454c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4552bd6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c160010 con 0x7f454c0cbef0 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4552bd6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c15f400 con 0x7f454c0cbb20 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4552bd6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c15f6e0 con 0x7f454c161230 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 0x7f454c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 0x7f454c0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50842/0 (socket says 172.21.15.83:50842) 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 -- 172.21.15.83:0/666262856 learned_addr learned my addr 172.21.15.83:0/666262856 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4550972700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c0cbb20 0x7f454c14d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4551173700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f454c161230 0x7f454c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 -- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f454c161230 msgr2=0x7f454c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f454c161230 0x7f454c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 -- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c0cbb20 msgr2=0x7f454c14d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c0cbb20 0x7f454c14d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 -- 172.21.15.83:0/666262856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45380094a0 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4550972700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c0cbb20 0x7f454c14d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f454bfff700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 0x7f454c0c95b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4538000c00 tx=0x7f4538004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380036a0 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538003800 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538003b70 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c0b26e0 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.348+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c0cd010 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.349+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c160b60 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.350+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f453800f050 con 0x7f454c0cbef0 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.350+0000 7f45497fa700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45300682f0 0x7f453006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.350+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f45380873d0 con 0x7f454c0cbef0 2023-11-28T11:09:51.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.350+0000 7f4550972700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45300682f0 0x7f453006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.350+0000 7f4550972700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45300682f0 0x7f453006a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f454000a9b0 tx=0x7f4540008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.354+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4538053ba0 con 0x7f454c0cbef0 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.609+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f454c074240 con 0x7f454c0cbef0 2023-11-28T11:09:51.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.612+0000 7f45497fa700 1 -- 172.21.15.83:0/666262856 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f45380571c0 con 0x7f454c0cbef0 2023-11-28T11:09:51.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45300682f0 msgr2=0x7f453006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45300682f0 0x7f453006a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f454000a9b0 tx=0x7f4540008040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 msgr2=0x7f454c0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 0x7f454c0c95b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4538000c00 tx=0x7f4538004750 comp rx=0 tx=0).stop 2023-11-28T11:09:51.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 shutdown_connections 2023-11-28T11:09:51.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f454c161230 0x7f454c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c0cbb20 0x7f454c14d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45300682f0 0x7f453006a7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 --2- 172.21.15.83:0/666262856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c0cbef0 0x7f454c0c95b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 >> 172.21.15.83:0/666262856 conn(0x7f454c0b2ca0 msgr2=0x7f454c15ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 shutdown_connections 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.614+0000 7f4552bd6700 1 -- 172.21.15.83:0/666262856 wait complete. 2023-11-28T11:09:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-28T11:09:51.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.746+0000 7f29e830b700 1 Processor -- start 2023-11-28T11:09:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.746+0000 7f29e830b700 1 -- start start 2023-11-28T11:09:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e830b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29e0161020 0x7f29e00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e830b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e00b1c60 0x7f29e006b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e830b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e00b20b0 con 0x7f29e00af030 2023-11-28T11:09:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e830b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e00cb820 con 0x7f29e0161020 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e830b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00ce620 con 0x7f29e00b1c60 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e58a6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29e0161020 0x7f29e00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e60a7700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f29e00af030 0x7f29e00cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48708/0 (socket says 172.21.15.83:48708) 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e60a7700 1 -- 172.21.15.83:0/3308334819 learned_addr learned my addr 172.21.15.83:0/3308334819 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e58a6700 1 --2- 172.21.15.83:0/3308334819 >> v2:172.21.15.83:3300/0 conn(0x7f29e0161020 0x7f29e00b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50850/0 (socket says 172.21.15.83:50850) 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e68a8700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e00b1c60 0x7f29e006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e58a6700 1 -- 172.21.15.83:0/3308334819 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e00b1c60 msgr2=0x7f29e006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e58a6700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e00b1c60 0x7f29e006b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e58a6700 1 -- 172.21.15.83:0/3308334819 >> v1:172.21.15.83:6789/0 conn(0x7f29e00af030 legacy=0x7f29e00cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e58a6700 1 -- 172.21.15.83:0/3308334819 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00ce8c0 con 0x7f29e0161020 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.747+0000 7f29e68a8700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e00b1c60 0x7f29e006b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.748+0000 7f29e58a6700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e0161020 0x7f29e00b1720 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009840 tx=0x7f29d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5028c34407b5967a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.748+0000 7f29e50a5700 1 -- 172.21.15.83:0/3308334819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0011830 con 0x7f29e0161020 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e50a5700 1 -- 172.21.15.83:0/3308334819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0011990 con 0x7f29e0161020 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e50a5700 1 -- 172.21.15.83:0/3308334819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0011d20 con 0x7f29e0161020 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 -- 172.21.15.83:0/3308334819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e0161020 msgr2=0x7f29e00b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e0161020 0x7f29e00b1720 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009840 tx=0x7f29d0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 -- 172.21.15.83:0/3308334819 shutdown_connections 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e00b1c60 0x7f29e006b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 --2- 172.21.15.83:0/3308334819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e0161020 0x7f29e00b1720 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 -- 172.21.15.83:0/3308334819 >> 172.21.15.83:0/3308334819 conn(0x7f29e00cc720 msgr2=0x7f29e00ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 -- 172.21.15.83:0/3308334819 shutdown_connections 2023-11-28T11:09:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.749+0000 7f29e830b700 1 -- 172.21.15.83:0/3308334819 wait complete. 2023-11-28T11:09:51.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.750+0000 7f29e830b700 1 Processor -- start 2023-11-28T11:09:51.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 -- start start 2023-11-28T11:09:51.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 0x7f29e0155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e00b1c60 0x7f29e01e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e01e6650 0x7f29e006b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00ac120 con 0x7f29e00af030 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0156060 con 0x7f29e00b1c60 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.751+0000 7f29e830b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e01e6b40 con 0x7f29e01e6650 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 0x7f29e0155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 0x7f29e0155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50854/0 (socket says 172.21.15.83:50854) 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 -- 172.21.15.83:0/3281104605 learned_addr learned my addr 172.21.15.83:0/3281104605 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e58a6700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e00b1c60 0x7f29e01e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e68a8700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e01e6650 0x7f29e006b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 -- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e01e6650 msgr2=0x7f29e006b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e01e6650 0x7f29e006b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 -- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e00b1c60 msgr2=0x7f29e01e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e00b1c60 0x7f29e01e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 -- 172.21.15.83:0/3281104605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00094a0 con 0x7f29e00af030 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e60a7700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 0x7f29e0155b20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c550 tx=0x7f29dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc005d70 con 0x7f29e00af030 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc0070d0 con 0x7f29e00af030 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e00cee80 con 0x7f29e00af030 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc007440 con 0x7f29e00af030 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.752+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e00ceba0 con 0x7f29e00af030 2023-11-28T11:09:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.753+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e00cf160 con 0x7f29e00af030 2023-11-28T11:09:51.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.753+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f29dc019030 con 0x7f29e00af030 2023-11-28T11:09:51.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.754+0000 7f29d6ffd700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29c40682f0 0x7f29c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.754+0000 7f29e58a6700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29c40682f0 0x7f29c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.754+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f29dc084e30 con 0x7f29e00af030 2023-11-28T11:09:51.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.754+0000 7f29e58a6700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29c40682f0 0x7f29c406a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009840 tx=0x7f29d001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.758+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f29dc04b5e0 con 0x7f29e00af030 2023-11-28T11:09:51.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.981+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29e00b20b0 con 0x7f29e00af030 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.982+0000 7f29d6ffd700 1 -- 172.21.15.83:0/3281104605 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f29e00b20b0 con 0x7f29e00af030 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29c40682f0 msgr2=0x7f29c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29c40682f0 0x7f29c406a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009840 tx=0x7f29d001c040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 msgr2=0x7f29e0155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 0x7f29e0155b20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c550 tx=0x7f29dc00c860 comp rx=0 tx=0).stop 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 shutdown_connections 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29e01e6650 0x7f29e006b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29c40682f0 0x7f29c406a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e00af030 0x7f29e0155b20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 --2- 172.21.15.83:0/3281104605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e00b1c60 0x7f29e01e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 >> 172.21.15.83:0/3281104605 conn(0x7f29e00cc720 msgr2=0x7f29e00afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 shutdown_connections 2023-11-28T11:09:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:51.984+0000 7f29e830b700 1 -- 172.21.15.83:0/3281104605 wait complete. 2023-11-28T11:09:51.987 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:09:52.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:09:46.728367+0000', 'modified': '2023-11-28T11:09:49.740814+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5098}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5098': {'gid': 5098, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.90:6833/686672772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6832', 'nonce': 686672772}, {'type': 'v1', 'addr': '172.21.15.90:6833', 'nonce': 686672772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:09:52.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:09:52.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:09:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.521+0000 7fbefa19e700 1 Processor -- start 2023-11-28T11:09:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.521+0000 7fbefa19e700 1 -- start start 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.522+0000 7fbefa19e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbef40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.522+0000 7fbefa19e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40cd240 con 0x7fbef40cddb0 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.522+0000 7fbefa19e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40cd920 con 0x7fbef415c390 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.522+0000 7fbefa19e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef406afc0 con 0x7fbef40cc8b0 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.522+0000 7fbef999d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbef40cddb0 0x7fbef40ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48726/0 (socket says 172.21.15.83:48726) 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.522+0000 7fbef999d700 1 -- 172.21.15.83:0/2182392679 learned_addr learned my addr 172.21.15.83:0/2182392679 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef899b700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbef40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073840409 0 0) 0x7fbeec001490 con 0x7fbef40cddb0 2023-11-28T11:09:52.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee00035b0 con 0x7fbef40cddb0 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046820328 0 0) 0x7fbee00035b0 con 0x7fbef40cddb0 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 msgr2=0x7fbef40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbef40ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 >> v1:172.21.15.90:6789/0 conn(0x7fbef415c390 legacy=0x7fbef40c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40cab50 con 0x7fbef40cddb0 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fbeec001b80 con 0x7fbef40cddb0 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 >> v1:172.21.15.83:6789/0 conn(0x7fbef40cddb0 legacy=0x7fbef40ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbee0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbee0004850 0x7fbee0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbee0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef406afc0 con 0x7fbee0003a30 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000bac0 con 0x7fbee0004850 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000bc50 con 0x7fbef40cc8b0 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbee0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef899b700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbee0004850 0x7fbee0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef999d700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbee0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 -- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 msgr2=0x7fbee0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbee0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 -- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbee0004850 msgr2=0x7fbee0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbee0004850 0x7fbee0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 -- 172.21.15.83:0/2182392679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00035b0 con 0x7fbee0003a30 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef999d700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef40cc8b0 0x7fbee0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.523+0000 7fbef919c700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbee0003e40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbee40017c0 tx=0x7fbee0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4ff93d4faa3dad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40052d0 con 0x7fbee0003a30 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee4005430 con 0x7fbee0003a30 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbeebfff700 1 -- 172.21.15.83:0/2182392679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40038c0 con 0x7fbee0003a30 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbefa19e700 1 -- 172.21.15.83:0/2182392679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 msgr2=0x7fbee0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbefa19e700 1 --2- 172.21.15.83:0/2182392679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbee0003e40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbee40017c0 tx=0x7fbee0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbef999d700 1 -- 172.21.15.83:0/2182392679 reap_dead start 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbefa19e700 1 -- 172.21.15.83:0/2182392679 shutdown_connections 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbefa19e700 1 -- 172.21.15.83:0/2182392679 >> 172.21.15.83:0/2182392679 conn(0x7fbef406de00 msgr2=0x7fbef406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbefa19e700 1 -- 172.21.15.83:0/2182392679 shutdown_connections 2023-11-28T11:09:52.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.524+0000 7fbefa19e700 1 -- 172.21.15.83:0/2182392679 wait complete. 2023-11-28T11:09:52.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 Processor -- start 2023-11-28T11:09:52.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 -- start start 2023-11-28T11:09:52.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbef4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbef41583b0 0x7fbef415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef415e430 0x7fbef415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415ede0 con 0x7fbee0003a30 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415c8e0 con 0x7fbef41583b0 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbefa19e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415cbc0 con 0x7fbef415e430 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbef919c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbef4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbef919c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbef4154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50874/0 (socket says 172.21.15.83:50874) 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.526+0000 7fbef919c700 1 -- 172.21.15.83:0/81168214 learned_addr learned my addr 172.21.15.83:0/81168214 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef999d700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef415e430 0x7fbef415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef899b700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbef41583b0 0x7fbef415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef919c700 1 -- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef415e430 msgr2=0x7fbef415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef919c700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef415e430 0x7fbef415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef919c700 1 -- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbef41583b0 msgr2=0x7fbef415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef919c700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbef41583b0 0x7fbef415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef919c700 1 -- 172.21.15.83:0/81168214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40cb110 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbef919c700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbef4154590 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c00 tx=0x7fbee4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbee9ffb700 1 -- 172.21.15.83:0/81168214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4007240 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbee9ffb700 1 -- 172.21.15.83:0/81168214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee40073a0 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbee9ffb700 1 -- 172.21.15.83:0/81168214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4001290 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40cae30 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.527+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef40af090 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.529+0000 7fbee9ffb700 1 -- 172.21.15.83:0/81168214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbee402e070 con 0x7fbee0003a30 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.529+0000 7fbee9ffb700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbed80682f0 0x7fbed806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.529+0000 7fbee9ffb700 1 -- 172.21.15.83:0/81168214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbee4086ba0 con 0x7fbee0003a30 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.529+0000 7fbef899b700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbed80682f0 0x7fbed806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.529+0000 7fbef899b700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbed80682f0 0x7fbed806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbef415e1f0 tx=0x7fbef0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.531+0000 7fbf07589b80 1 Processor -- start 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.531+0000 7fbf07589b80 1 -- start start 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c7767f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c7767eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbf07589b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7766b1bc0 con 0x55c7767c18c0 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c776720350 con 0x55c7767f1a70 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c776719880 con 0x55c7767f2380 2023-11-28T11:09:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c7767f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c7767f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50884/0 (socket says 172.21.15.83:50884) 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 -- 172.21.15.83:0/2543315590 learned_addr learned my addr 172.21.15.83:0/2543315590 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef919c700 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef999d700 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c7767eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 -- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 msgr2=0x55c7767eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c7767eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 -- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 msgr2=0x55c7767c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 -- 172.21.15.83:0/2543315590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c77659b5e0 con 0x55c7767c18c0 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.532+0000 7fbef899b700 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c7767f1e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbef0014bd0 tx=0x7fbef0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497e0e613fcd1d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbed6ffd700 1 -- 172.21.15.83:0/2543315590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0014ee0 con 0x55c7767c18c0 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbed6ffd700 1 -- 172.21.15.83:0/2543315590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0016ad0 con 0x55c7767c18c0 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbed6ffd700 1 -- 172.21.15.83:0/2543315590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0016e40 con 0x55c7767c18c0 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 -- 172.21.15.83:0/2543315590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 msgr2=0x55c7767f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c7767f1e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbef0014bd0 tx=0x7fbef0017000 comp rx=0 tx=0).stop 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 -- 172.21.15.83:0/2543315590 shutdown_connections 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c7767eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 --2- 172.21.15.83:0/2543315590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c7767f1e40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 -- 172.21.15.83:0/2543315590 >> 172.21.15.83:0/2543315590 conn(0x55c7767c4fc0 msgr2=0x55c7767c5950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 -- 172.21.15.83:0/2543315590 shutdown_connections 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.533+0000 7fbf07589b80 1 -- 172.21.15.83:0/2543315590 wait complete. 2023-11-28T11:09:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.534+0000 7fbf07589b80 1 -- start start 2023-11-28T11:09:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.534+0000 7fbf07589b80 1 Processor -- start 2023-11-28T11:09:52.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c77681c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f1a70 0x55c7767f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f2380 0x55c7767f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77659d8d0 con 0x55c7767c18c0 2023-11-28T11:09:52.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c776803ec0 con 0x55c7767f2380 2023-11-28T11:09:52.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c77680f530 con 0x55c7767f1a70 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 10 client.0 _ll_register_callbacks cb 0x55c7765bf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c77681c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c77681c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50888/0 (socket says 172.21.15.83:50888) 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 -- 172.21.15.83:0/3251166729 learned_addr learned my addr 172.21.15.83:0/3251166729 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef999d700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f2380 0x55c7767f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef899b700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f1a70 0x55c7767f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 -- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f1a70 msgr2=0x55c7767f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f1a70 0x55c7767f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 -- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f2380 msgr2=0x55c7767f08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f2380 0x55c7767f08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 -- 172.21.15.83:0/3251166729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef0014880 con 0x55c7767c18c0 2023-11-28T11:09:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbef919c700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c77681c800 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbee4056eb0 tx=0x7fbee4056bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5345103ae9463e11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbecffff700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbecffff700 1 -- 172.21.15.83:0/3251166729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4056470 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbecffff700 1 -- 172.21.15.83:0/3251166729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbee4080dd0 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.535+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7766e7140 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbecffff700 1 -- 172.21.15.83:0/3251166729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbee40565d0 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 10 client.5490 fetch_fsmap learned FSMap version 17 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55c776547d90 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbecffff700 1 -- 172.21.15.83:0/3251166729 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7766e7140 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbecffff700 1 -- 172.21.15.83:0/3251166729 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbee405eb80 con 0x55c7767c18c0 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 10 client.5490 fetch_fsmap finished waiting for FSMap version 17 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 10 client.5490 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:09:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 20 client.5490 populate_metadata read hostname 'smithi083' 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c77680fe50 0x55c776810220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 4 client.5490 mds_command: new command op to 5098 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 --> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c77659df10 con 0x55c77680fe50 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.536+0000 7fbef899b700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c77680fe50 0x55c776810220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbecffff700 10 client.5490 ms_handle_connect on v2:172.21.15.90:6832/686672772 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbef899b700 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c77680fe50 0x55c776810220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e266968270afc1f1 server_cookie=501a51081f13dd13 in_seq=0 out_seq=0 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbecffff700 1 -- 172.21.15.83:0/3251166729 <== mds.0 v2:172.21.15.90:6832/686672772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c77659df10 con 0x55c77680fe50 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbecffff700 10 client.5490 handle_command_reply: tid=0 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 client.5490 shutdown 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 msgr2=0x55c77681c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c77681c800 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbee4056eb0 tx=0x7fbee4056bf0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 shutdown_connections 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c77680fe50 0x55c776810220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f2380 0x55c7767f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbecffff700 0 client.5490 ms_handle_reset on v2:172.21.15.90:6832/686672772 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767c18c0 0x55c77681c800 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3251166729 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f1a70 0x55c7767f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.537+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 >> 172.21.15.83:0/3251166729 conn(0x55c7767c4fc0 msgr2=0x55c7767c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.538+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 shutdown_connections 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.538+0000 7fbf07589b80 1 -- 172.21.15.83:0/3251166729 wait complete. 2023-11-28T11:09:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.538+0000 7fbf07589b80 20 client.5490 trim_cache size 0 max 16384 2023-11-28T11:09:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 Processor -- start 2023-11-28T11:09:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 -- start start 2023-11-28T11:09:52.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767c18c0 0x55c7767c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 0x55c77680e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c77680e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7767ec0a0 con 0x55c7767f1a70 2023-11-28T11:09:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7767c7430 con 0x55c7767c18c0 2023-11-28T11:09:52.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7767eb2d0 con 0x55c7767f2380 2023-11-28T11:09:52.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbef899b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 0x55c77680e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.555+0000 7fbef899b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 0x55c77680e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50898/0 (socket says 172.21.15.83:50898) 2023-11-28T11:09:52.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 -- 172.21.15.83:0/3765389219 learned_addr learned my addr 172.21.15.83:0/3765389219 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 -- 172.21.15.83:0/3765389219 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 msgr2=0x55c77680e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c77680e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 -- 172.21.15.83:0/3765389219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767c18c0 msgr2=0x55c7767c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767c18c0 0x55c7767c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 -- 172.21.15.83:0/3765389219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7766e0140 con 0x55c7767f1a70 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbef899b700 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 0x55c77680e340 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbef0008010 tx=0x7fbef001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47148d8f106267d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbed77fe700 1 -- 172.21.15.83:0/3765389219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0031070 con 0x55c7767f1a70 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbed77fe700 1 -- 172.21.15.83:0/3765389219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef00332d0 con 0x55c7767f1a70 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbed77fe700 1 -- 172.21.15.83:0/3765389219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0014ee0 con 0x55c7767f1a70 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 -- 172.21.15.83:0/3765389219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 msgr2=0x55c77680e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 0x55c77680e340 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbef0008010 tx=0x7fbef001b000 comp rx=0 tx=0).stop 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 -- 172.21.15.83:0/3765389219 shutdown_connections 2023-11-28T11:09:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767f2380 0x55c77680e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767c18c0 0x55c7767c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3765389219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f1a70 0x55c77680e340 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.556+0000 7fbf07589b80 1 -- 172.21.15.83:0/3765389219 >> 172.21.15.83:0/3765389219 conn(0x55c7767c4fc0 msgr2=0x55c77680eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.557+0000 7fbf07589b80 1 -- 172.21.15.83:0/3765389219 shutdown_connections 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.557+0000 7fbf07589b80 1 -- 172.21.15.83:0/3765389219 wait complete. 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.557+0000 7fbf07589b80 1 -- start start 2023-11-28T11:09:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.557+0000 7fbf07589b80 1 Processor -- start 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767c18c0 0x55c776810220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767eca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 0x55c7767ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7767f2980 con 0x55c7767f2380 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c776875c80 con 0x55c7767f1a70 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7767ec8a0 con 0x55c7767c18c0 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbf07589b80 10 client.0 _ll_register_callbacks cb 0x55c7765bf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 0x55c7767ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 0x55c7767ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50906/0 (socket says 172.21.15.83:50906) 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 -- 172.21.15.83:0/3700033318 learned_addr learned my addr 172.21.15.83:0/3700033318 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef919c700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767c18c0 0x55c776810220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef899b700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767eca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 -- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767c18c0 msgr2=0x55c776810220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767c18c0 0x55c776810220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 -- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 msgr2=0x55c7767eca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767eca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.558+0000 7fbef999d700 1 -- 172.21.15.83:0/3700033318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c77659b5e0 con 0x55c7767f2380 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbef999d700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 0x55c7767ecfc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbef40c69c0 tx=0x7fbeec001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79295c613076d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbecffff700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbecffff700 1 -- 172.21.15.83:0/3700033318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec015990 con 0x55c7767f2380 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbecffff700 1 -- 172.21.15.83:0/3700033318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbeec015af0 con 0x55c7767f2380 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7766d7160 con 0x55c7767f2380 2023-11-28T11:09:52.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbecffff700 1 -- 172.21.15.83:0/3700033318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeec015cb0 con 0x55c7767f2380 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 10 client.5496 fetch_fsmap learned FSMap version 17 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55c77656e200 con 0x55c7767f2380 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbecffff700 1 -- 172.21.15.83:0/3700033318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7766d7160 con 0x55c7767f2380 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbecffff700 1 -- 172.21.15.83:0/3700033318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbeec018470 con 0x55c7767f2380 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 10 client.5496 fetch_fsmap finished waiting for FSMap version 17 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 10 client.5496 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 20 client.5496 populate_metadata read hostname 'smithi083' 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c776802880 0x55c776802c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 4 client.5496 mds_command: new command op to 5098 tid=0[{"prefix": "status"}] 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 --> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c7766c1170 con 0x55c776802880 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.559+0000 7fbef919c700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c776802880 0x55c776802c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbecffff700 10 client.5496 ms_handle_connect on v2:172.21.15.90:6832/686672772 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbef919c700 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c776802880 0x55c776802c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4930a87c0e497984 server_cookie=3f6e180b2c219a30 in_seq=0 out_seq=0 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbecffff700 1 -- 172.21.15.83:0/3700033318 <== mds.0 v2:172.21.15.90:6832/686672772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c7766c1170 con 0x55c776802880 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbecffff700 10 client.5496 handle_command_reply: tid=0 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 client.5496 shutdown 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 msgr2=0x55c7767ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 0x55c7767ecfc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbef40c69c0 tx=0x7fbeec001b90 comp rx=0 tx=0).stop 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 shutdown_connections 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:6832/686672772,v1:172.21.15.90:6833/686672772] conn(0x55c776802880 0x55c776802c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7767f2380 0x55c7767ecfc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbecffff700 0 client.5496 ms_handle_reset on v2:172.21.15.90:6832/686672772 2023-11-28T11:09:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c7767c18c0 0x55c776810220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 --2- 172.21.15.83:0/3700033318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7767f1a70 0x55c7767eca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 >> 172.21.15.83:0/3700033318 conn(0x55c7767c4fc0 msgr2=0x55c77680e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 shutdown_connections 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 1 -- 172.21.15.83:0/3700033318 wait complete. 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.560+0000 7fbf07589b80 20 client.5496 trim_cache size 0 max 16384 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "69424f07-94c3-4dbc-8507-3551775e0a09", 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-28T11:09:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5098, 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8306242340000001, 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 17, 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 39, 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 39, 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 13.270965690000001 2023-11-28T11:09:52.573 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbed80682f0 msgr2=0x7fbed806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbed80682f0 0x7fbed806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbef415e1f0 tx=0x7fbef0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 msgr2=0x7fbef4154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbef4154590 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c00 tx=0x7fbee4001120 comp rx=0 tx=0).stop 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 shutdown_connections 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbef415e430 0x7fbef415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbed80682f0 0x7fbed806a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee0003a30 0x7fbef4154590 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 --2- 172.21.15.83:0/81168214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbef41583b0 0x7fbef415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 >> 172.21.15.83:0/81168214 conn(0x7fbef406de00 msgr2=0x7fbef40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 shutdown_connections 2023-11-28T11:09:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.561+0000 7fbefa19e700 1 -- 172.21.15.83:0/81168214 wait complete. 2023-11-28T11:09:52.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.694+0000 7f0d16664700 1 Processor -- start 2023-11-28T11:09:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 -- start start 2023-11-28T11:09:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0d1015a510 0x7f0d100cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d100cfdf0 0x7f0d100b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d1015a8e0 0x7f0d100b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d100cd780 con 0x7f0d1015a8e0 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1015f1d0 con 0x7f0d1015a510 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.695+0000 7f0d16664700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015eef0 con 0x7f0d100cfdf0 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d1015a8e0 0x7f0d100b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d0ffff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0d1015a510 0x7f0d100cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d1015a8e0 0x7f0d100b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50914/0 (socket says 172.21.15.83:50914) 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d0f7fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d100cfdf0 0x7f0d100b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 -- 172.21.15.83:0/2581858659 learned_addr learned my addr 172.21.15.83:0/2581858659 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 -- 172.21.15.83:0/2581858659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d100cfdf0 msgr2=0x7f0d100b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 --2- 172.21.15.83:0/2581858659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d100cfdf0 0x7f0d100b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 -- 172.21.15.83:0/2581858659 >> v2:172.21.15.90:3300/0 conn(0x7f0d1015a510 msgr2=0x7f0d100cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 --2- 172.21.15.83:0/2581858659 >> v2:172.21.15.90:3300/0 conn(0x7f0d1015a510 0x7f0d100cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 -- 172.21.15.83:0/2581858659 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100cedd0 con 0x7f0d1015a8e0 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d14c01700 1 --2- 172.21.15.83:0/2581858659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d100b2ea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d100ce6b0 tx=0x7f0d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b04ae9894e8a8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d0effd700 1 -- 172.21.15.83:0/2581858659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800e070 con 0x7f0d1015a8e0 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d0effd700 1 -- 172.21.15.83:0/2581858659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0800b910 con 0x7f0d1015a8e0 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.696+0000 7f0d0effd700 1 -- 172.21.15.83:0/2581858659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800bcc0 con 0x7f0d1015a8e0 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 -- 172.21.15.83:0/2581858659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 msgr2=0x7f0d100b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 --2- 172.21.15.83:0/2581858659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d100b2ea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d100ce6b0 tx=0x7f0d0800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 -- 172.21.15.83:0/2581858659 shutdown_connections 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 --2- 172.21.15.83:0/2581858659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d100b2ea0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 --2- 172.21.15.83:0/2581858659 >> v2:172.21.15.90:3300/0 conn(0x7f0d1015a510 0x7f0d100cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 --2- 172.21.15.83:0/2581858659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d100cfdf0 0x7f0d100b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 -- 172.21.15.83:0/2581858659 >> 172.21.15.83:0/2581858659 conn(0x7f0d100cd960 msgr2=0x7f0d100cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 -- 172.21.15.83:0/2581858659 shutdown_connections 2023-11-28T11:09:52.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.697+0000 7f0d16664700 1 -- 172.21.15.83:0/2581858659 wait complete. 2023-11-28T11:09:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.698+0000 7f0d16664700 1 Processor -- start 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.698+0000 7f0d16664700 1 -- start start 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d16664700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d100cfdf0 0x7f0d100c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d16664700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d1015a510 0x7f0d1015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d16664700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d1015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d16664700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015eb70 con 0x7f0d1015a8e0 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d16664700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015e240 con 0x7f0d100cfdf0 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d16664700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015e520 con 0x7f0d1015a510 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d1015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d1015d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50920/0 (socket says 172.21.15.83:50920) 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 -- 172.21.15.83:0/1302436312 learned_addr learned my addr 172.21.15.83:0/1302436312 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d0ffff700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d100cfdf0 0x7f0d100c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d0f7fe700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d1015a510 0x7f0d1015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 -- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d1015a510 msgr2=0x7f0d1015b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d1015a510 0x7f0d1015b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 -- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d100cfdf0 msgr2=0x7f0d100c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d100cfdf0 0x7f0d100c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.699+0000 7f0d14c01700 1 -- 172.21.15.83:0/1302436312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08009b30 con 0x7f0d1015a8e0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.700+0000 7f0d14c01700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d1015d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d08012000 tx=0x7f0d08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.700+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080099b0 con 0x7f0d1015a8e0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.700+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0800ce80 con 0x7f0d1015a8e0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.700+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100cf390 con 0x7f0d1015a8e0 2023-11-28T11:09:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.700+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08004490 con 0x7f0d1015a8e0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.700+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d100cf0b0 con 0x7f0d1015a8e0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.701+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d100b3d30 con 0x7f0d1015a8e0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.701+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d080045f0 con 0x7f0d1015a8e0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.701+0000 7f0d0cff9700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0cf00682f0 0x7f0cf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.701+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0d08087260 con 0x7f0d1015a8e0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.701+0000 7f0d0ffff700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0cf00682f0 0x7f0cf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.702+0000 7f0d0ffff700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0cf00682f0 0x7f0cf006a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009be0 tx=0x7f0d00009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.705+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d08053a30 con 0x7f0d1015a8e0 2023-11-28T11:09:52.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.931+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0d1015ecb0 con 0x7f0d1015a8e0 2023-11-28T11:09:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.931+0000 7f0d0cff9700 1 -- 172.21.15.83:0/1302436312 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0d08020060 con 0x7f0d1015a8e0 2023-11-28T11:09:52.933 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0cf00682f0 msgr2=0x7f0cf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0cf00682f0 0x7f0cf006a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009be0 tx=0x7f0d00009210 comp rx=0 tx=0).stop 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 msgr2=0x7f0d1015d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d1015d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d08012000 tx=0x7f0d08004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 shutdown_connections 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0cf00682f0 0x7f0cf006a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1015a8e0 0x7f0d1015d330 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d100cfdf0 0x7f0d100c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 --2- 172.21.15.83:0/1302436312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0d1015a510 0x7f0d1015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 >> 172.21.15.83:0/1302436312 conn(0x7f0d100cd960 msgr2=0x7f0d100b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 shutdown_connections 2023-11-28T11:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:52.933+0000 7f0d16664700 1 -- 172.21.15.83:0/1302436312 wait complete. 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:52.947 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:52.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:52.948 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-28T11:09:52.965 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:52.965 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:52.966 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42281sec preferred_lft 42281sec 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::22f1:9e4c:955b:b4a/64 scope link noprefixroute 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::80fe:9fff:fe99:6cb2/64 scope link 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.967 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::1c9b:f5ff:fe74:5abe/64 scope link 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:d1:52:12:18:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5cd1:52ff:fe12:1849/64 scope link 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:52.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:52.968 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:53.030 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:09:53.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.032 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-28T11:09:53.093 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-28T11:09:53.093 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:53.093 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:09:53.093 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:09:53.093 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:53.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.094 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:09:53.094 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:09:53.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:09:53.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:09:53.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:09:53.094 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:09:53.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.197 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:09:53.215 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:09:53.215 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:09:53.215 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:53.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-28T11:09:53.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:53.282 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:53.339 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:53.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:53.340 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:53.395 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-28T11:09:53.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:53.480 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:53.480 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:53.503 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:53.503 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:53.504 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:53.504 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:53.504 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:53.504 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:53.505 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:53.505 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:53.579 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:53.580 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:09:53.581 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:09:53.581 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:09:53.582 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:09:53.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:53.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.609 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:53.696 INFO:teuthology.orchestra.run.smithi083.stdout:client4979 2023-11-28T11:09:53.696 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:53.697 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:53.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.714 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T11:09:53.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.778 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T11:09:53.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.837 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T11:09:53.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.896 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T11:09:53.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.962 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T11:09:54.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:54.023 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T11:09:54.085 INFO:teuthology.orchestra.run.smithi083.stdout:./ 2023-11-28T11:09:54.086 INFO:teuthology.orchestra.run.smithi083.stdout:./rfile 2023-11-28T11:09:54.086 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir 2023-11-28T11:09:54.086 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir/pfile 2023-11-28T11:09:54.086 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir/childdir 2023-11-28T11:09:54.086 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir/childdir/cfile 2023-11-28T11:09:54.087 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:54.087 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:09:54.087 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:09:54.087 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:09:54.087 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T11:09:54.087 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:09:54.191 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:09:54.381 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:54.382 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:09:54.383 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:09:54.385 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:09:54.385 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T11:09:54.385 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:09:54.487 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627778 2023-11-28T11:09:54.676 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:54.677 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:09:54.677 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:09:54.677 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:09:54.677 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T11:09:54.677 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:09:54.786 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2023-11-28T11:09:54.975 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:54.975 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:09:54.975 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:09:54.976 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:09:54.976 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T11:09:54.976 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:09:55.083 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628279 2023-11-28T11:09:55.272 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:55.272 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:09:55.272 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:09:55.272 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:09:55.273 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T11:09:55.273 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:09:55.378 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2023-11-28T11:09:55.567 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:55.568 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:09:55.568 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:09:55.568 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:09:55.568 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T11:09:55.568 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:09:55.673 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628280 2023-11-28T11:09:55.862 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:55.862 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:55.920 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:09:55.920 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:55.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:56.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.138+0000 7fe3a9c2e700 1 Processor -- start 2023-11-28T11:09:56.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.139+0000 7fe3a9c2e700 1 -- start start 2023-11-28T11:09:56.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a9c2e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a480 0x7fe3a40cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a9c2e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 0x7fe3a40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a9c2e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a415efa0 con 0x7fe3a415a850 2023-11-28T11:09:56.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a9c2e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a40ceab0 con 0x7fe3a40cfce0 2023-11-28T11:09:56.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a9c2e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40cecc0 con 0x7fe3a415a480 2023-11-28T11:09:56.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a3fff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 0x7fe3a40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a480 0x7fe3a40cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a3fff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 0x7fe3a40b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:37922/0 (socket says 172.21.15.83:37922) 2023-11-28T11:09:56.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a3fff700 1 -- 172.21.15.83:0/336223063 learned_addr learned my addr 172.21.15.83:0/336223063 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:56.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a37fe700 1 -- 172.21.15.83:0/336223063 >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 msgr2=0x7fe3a40b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a37fe700 1 --2- 172.21.15.83:0/336223063 >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 0x7fe3a40b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a37fe700 1 -- 172.21.15.83:0/336223063 >> v1:172.21.15.90:6789/0 conn(0x7fe3a415a850 legacy=0x7fe3a40b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a37fe700 1 -- 172.21.15.83:0/336223063 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40cefa0 con 0x7fe3a415a480 2023-11-28T11:09:56.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.140+0000 7fe3a3fff700 1 --2- 172.21.15.83:0/336223063 >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 0x7fe3a40b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:56.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a37fe700 1 --2- 172.21.15.83:0/336223063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a480 0x7fe3a40cf7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe394009d90 tx=0x7fe39400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59ea6d3af3e835d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a27fc700 1 -- 172.21.15.83:0/336223063 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400e070 con 0x7fe3a415a480 2023-11-28T11:09:56.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a27fc700 1 -- 172.21.15.83:0/336223063 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39400bb20 con 0x7fe3a415a480 2023-11-28T11:09:56.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a27fc700 1 -- 172.21.15.83:0/336223063 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394012430 con 0x7fe3a415a480 2023-11-28T11:09:56.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/336223063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a480 msgr2=0x7fe3a40cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/336223063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a480 0x7fe3a40cf7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe394009d90 tx=0x7fe39400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/336223063 shutdown_connections 2023-11-28T11:09:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/336223063 >> v2:172.21.15.90:3300/0 conn(0x7fe3a40cfce0 0x7fe3a40b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/336223063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a480 0x7fe3a40cf7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/336223063 >> 172.21.15.83:0/336223063 conn(0x7fe3a406de20 msgr2=0x7fe3a406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/336223063 shutdown_connections 2023-11-28T11:09:56.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.141+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/336223063 wait complete. 2023-11-28T11:09:56.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.143+0000 7fe3a9c2e700 1 Processor -- start 2023-11-28T11:09:56.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.143+0000 7fe3a9c2e700 1 -- start start 2023-11-28T11:09:56.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a9c2e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 0x7fe3a41ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a9c2e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a415a480 0x7fe3a40c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a9c2e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a850 0x7fe3a40ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a9c2e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40a7740 con 0x7fe3a40cfce0 2023-11-28T11:09:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a9c2e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41efd80 con 0x7fe3a415a480 2023-11-28T11:09:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a9c2e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40c9410 con 0x7fe3a415a850 2023-11-28T11:09:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 0x7fe3a41ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 0x7fe3a41ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50934/0 (socket says 172.21.15.83:50934) 2023-11-28T11:09:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a37fe700 1 -- 172.21.15.83:0/2927521126 learned_addr learned my addr 172.21.15.83:0/2927521126 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a2ffd700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a415a480 0x7fe3a40c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.144+0000 7fe3a3fff700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a850 0x7fe3a40ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a37fe700 1 -- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a850 msgr2=0x7fe3a40ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a37fe700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a850 0x7fe3a40ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a37fe700 1 -- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a415a480 msgr2=0x7fe3a40c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a37fe700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a415a480 0x7fe3a40c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a37fe700 1 -- 172.21.15.83:0/2927521126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c009550 con 0x7fe3a40cfce0 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a3fff700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a850 0x7fe3a40ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a37fe700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 0x7fe3a41ef840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe394014040 tx=0x7fe394004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394012430 con 0x7fe3a40cfce0 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394020070 con 0x7fe3a40cfce0 2023-11-28T11:09:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe394009a70 con 0x7fe3a40cfce0 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394004280 con 0x7fe3a40cfce0 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.145+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a40b4440 con 0x7fe3a40cfce0 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.146+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a40cf280 con 0x7fe3a40cfce0 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.146+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe39402e070 con 0x7fe3a40cfce0 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.147+0000 7fe39bfff700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3840682f0 0x7fe38406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.147+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe394022070 con 0x7fe3a40cfce0 2023-11-28T11:09:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.147+0000 7fe3a2ffd700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3840682f0 0x7fe38406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.147+0000 7fe3a2ffd700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3840682f0 0x7fe38406a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40aa770 tx=0x7fe38c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.151+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe394054230 con 0x7fe3a40cfce0 2023-11-28T11:09:56.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.334+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3a40b45d0 con 0x7fe3a40cfce0 2023-11-28T11:09:56.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.335+0000 7fe39bfff700 1 -- 172.21.15.83:0/2927521126 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fe394057850 con 0x7fe3a40cfce0 2023-11-28T11:09:56.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:56.336 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:47.730175+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3840682f0 msgr2=0x7fe38406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3840682f0 0x7fe38406a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40aa770 tx=0x7fe38c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 msgr2=0x7fe3a41ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 0x7fe3a41ef840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe394014040 tx=0x7fe394004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 shutdown_connections 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3a415a850 0x7fe3a40ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3840682f0 0x7fe38406a7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfce0 0x7fe3a41ef840 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 --2- 172.21.15.83:0/2927521126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a415a480 0x7fe3a40c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 >> 172.21.15.83:0/2927521126 conn(0x7fe3a406de20 msgr2=0x7fe3a406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 shutdown_connections 2023-11-28T11:09:56.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.337+0000 7fe3a9c2e700 1 -- 172.21.15.83:0/2927521126 wait complete. 2023-11-28T11:09:56.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:09:44.997179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:09:45.687674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:56.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:56.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.473+0000 7fc73f67d700 1 Processor -- start 2023-11-28T11:09:56.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.473+0000 7fc73f67d700 1 -- start start 2023-11-28T11:09:56.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73f67d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380aefd0 0x7fc7380d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73f67d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7380cd1c0 0x7fc7380cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73f67d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380b4bb0 con 0x7fc7380ccb60 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73f67d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7380d0af0 con 0x7fc7380cd1c0 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73f67d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73815ac30 con 0x7fc7380aefd0 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73cc18700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7380cd1c0 0x7fc7380cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73dc1a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc7380ccb60 0x7fc73815e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48806/0 (socket says 172.21.15.83:48806) 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.474+0000 7fc73dc1a700 1 -- 172.21.15.83:0/3741855285 learned_addr learned my addr 172.21.15.83:0/3741855285 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73d419700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380aefd0 0x7fc7380d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73cc18700 1 -- 172.21.15.83:0/3741855285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380aefd0 msgr2=0x7fc7380d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73cc18700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380aefd0 0x7fc7380d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73cc18700 1 -- 172.21.15.83:0/3741855285 >> v1:172.21.15.83:6789/0 conn(0x7fc7380ccb60 legacy=0x7fc73815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc72ffff700 1 -- 172.21.15.83:0/3741855285 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655438555 0 0) 0x7fc7380b4bb0 con 0x7fc7380ccb60 2023-11-28T11:09:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73cc18700 1 -- 172.21.15.83:0/3741855285 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380ccfa0 con 0x7fc7380cd1c0 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73d419700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380aefd0 0x7fc7380d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73cc18700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380cd1c0 0x7fc7380cc620 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc734009840 tx=0x7fc734009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86edd17ab2f3998 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc72ffff700 1 -- 172.21.15.83:0/3741855285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734011830 con 0x7fc7380cd1c0 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc72ffff700 1 -- 172.21.15.83:0/3741855285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc734011990 con 0x7fc7380cd1c0 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc72ffff700 1 -- 172.21.15.83:0/3741855285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734011d20 con 0x7fc7380cd1c0 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73f67d700 1 -- 172.21.15.83:0/3741855285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380cd1c0 msgr2=0x7fc7380cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.475+0000 7fc73f67d700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380cd1c0 0x7fc7380cc620 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc734009840 tx=0x7fc734009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.476+0000 7fc73f67d700 1 -- 172.21.15.83:0/3741855285 shutdown_connections 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.476+0000 7fc73f67d700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380aefd0 0x7fc7380d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.476+0000 7fc73f67d700 1 --2- 172.21.15.83:0/3741855285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380cd1c0 0x7fc7380cc620 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.476+0000 7fc73f67d700 1 -- 172.21.15.83:0/3741855285 >> 172.21.15.83:0/3741855285 conn(0x7fc7380b2b60 msgr2=0x7fc7380b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.476+0000 7fc73f67d700 1 -- 172.21.15.83:0/3741855285 shutdown_connections 2023-11-28T11:09:56.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.476+0000 7fc73f67d700 1 -- 172.21.15.83:0/3741855285 wait complete. 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.477+0000 7fc73f67d700 1 Processor -- start 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.477+0000 7fc73f67d700 1 -- start start 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.477+0000 7fc73f67d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 0x7fc7381f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.477+0000 7fc73f67d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380ccb60 0x7fc7380c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73f67d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7380cd1c0 0x7fc7381f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73f67d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381f0e60 con 0x7fc7380aefd0 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73f67d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381f0490 con 0x7fc7380cd1c0 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73f67d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381efea0 con 0x7fc7380ccb60 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 0x7fc7381f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 0x7fc7381f0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50948/0 (socket says 172.21.15.83:50948) 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 -- 172.21.15.83:0/2496125510 learned_addr learned my addr 172.21.15.83:0/2496125510 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73dc1a700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7380cd1c0 0x7fc7381f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73cc18700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380ccb60 0x7fc7380c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 -- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380ccb60 msgr2=0x7fc7380c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380ccb60 0x7fc7380c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 -- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7380cd1c0 msgr2=0x7fc7381f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7380cd1c0 0x7fc7381f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 -- 172.21.15.83:0/2496125510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340094a0 con 0x7fc7380aefd0 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73cc18700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380ccb60 0x7fc7380c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73d419700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 0x7fc7381f0920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c820 tx=0x7fc72800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728005c20 con 0x7fc7380aefd0 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728005d80 con 0x7fc7380aefd0 2023-11-28T11:09:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7380b50e0 con 0x7fc7380aefd0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.478+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728011820 con 0x7fc7380aefd0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.479+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7380b4e00 con 0x7fc7380aefd0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.479+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7380b53c0 con 0x7fc7380aefd0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.480+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc728020070 con 0x7fc7380aefd0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.480+0000 7fc72dffb700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc71c0682f0 0x7fc71c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.480+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc728085f90 con 0x7fc7380aefd0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.480+0000 7fc73cc18700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc71c0682f0 0x7fc71c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.480+0000 7fc73cc18700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc71c0682f0 0x7fc71c06a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc734014040 tx=0x7fc734008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.484+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7280526b0 con 0x7fc7380aefd0 2023-11-28T11:09:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.667+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7380b37b0 con 0x7fc7380aefd0 2023-11-28T11:09:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.667+0000 7fc72dffb700 1 -- 172.21.15.83:0/2496125510 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc728055cd0 con 0x7fc7380aefd0 2023-11-28T11:09:56.668 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:09:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.669+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc71c0682f0 msgr2=0x7fc71c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.669+0000 7fc73f67d700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc71c0682f0 0x7fc71c06a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc734014040 tx=0x7fc734008030 comp rx=0 tx=0).stop 2023-11-28T11:09:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.669+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 msgr2=0x7fc7381f0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.669+0000 7fc73f67d700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 0x7fc7381f0920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c820 tx=0x7fc72800cbe0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 shutdown_connections 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7380cd1c0 0x7fc7381f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc71c0682f0 0x7fc71c06a7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7380aefd0 0x7fc7381f0920 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 --2- 172.21.15.83:0/2496125510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc7380ccb60 0x7fc7380c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 >> 172.21.15.83:0/2496125510 conn(0x7fc7380b2b60 msgr2=0x7fc7380ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 shutdown_connections 2023-11-28T11:09:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.670+0000 7fc73f67d700 1 -- 172.21.15.83:0/2496125510 wait complete. 2023-11-28T11:09:56.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:56.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.804+0000 7fb600dd2700 1 Processor -- start 2023-11-28T11:09:56.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.804+0000 7fb600dd2700 1 -- start start 2023-11-28T11:09:56.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb600dd2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb5fc0af040 0x7fb5fc0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb600dd2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc06f610 0x7fb5fc0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb600dd2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc0cd6a0 con 0x7fb5fc06c500 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb600dd2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0cce00 con 0x7fb5fc0af040 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb600dd2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0cd0e0 con 0x7fb5fc06f610 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fa59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb5fc0af040 0x7fb5fc0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5f9d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb5fc06c500 0x7fb5fc06ef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48818/0 (socket says 172.21.15.83:48818) 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fa59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb5fc0af040 0x7fb5fc0cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50956/0 (socket says 172.21.15.83:50956) 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5f9d9b700 1 -- 172.21.15.83:0/3475410510 learned_addr learned my addr 172.21.15.83:0/3475410510 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:56.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fad9d700 1 --2- 172.21.15.83:0/3475410510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc06f610 0x7fb5fc0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fa59c700 1 -- 172.21.15.83:0/3475410510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc06f610 msgr2=0x7fb5fc0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fa59c700 1 --2- 172.21.15.83:0/3475410510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc06f610 0x7fb5fc0ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fa59c700 1 -- 172.21.15.83:0/3475410510 >> v1:172.21.15.83:6789/0 conn(0x7fb5fc06c500 legacy=0x7fb5fc06ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5fa59c700 1 -- 172.21.15.83:0/3475410510 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0cd3c0 con 0x7fb5fc0af040 2023-11-28T11:09:56.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.805+0000 7fb5f959a700 1 -- 172.21.15.83:0/3475410510 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499427088 0 0) 0x7fb5fc0cd6a0 con 0x7fb5fc06c500 2023-11-28T11:09:56.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb5fa59c700 1 --2- 172.21.15.83:0/3475410510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc0af040 0x7fb5fc0cc5d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009840 tx=0x7fb5f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53f77805e831a58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb5f959a700 1 -- 172.21.15.83:0/3475410510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0011830 con 0x7fb5fc0af040 2023-11-28T11:09:56.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb5f959a700 1 -- 172.21.15.83:0/3475410510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0011990 con 0x7fb5fc0af040 2023-11-28T11:09:56.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb5f959a700 1 -- 172.21.15.83:0/3475410510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0011d20 con 0x7fb5fc0af040 2023-11-28T11:09:56.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 -- 172.21.15.83:0/3475410510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc0af040 msgr2=0x7fb5fc0cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 --2- 172.21.15.83:0/3475410510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc0af040 0x7fb5fc0cc5d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009840 tx=0x7fb5f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 -- 172.21.15.83:0/3475410510 shutdown_connections 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 --2- 172.21.15.83:0/3475410510 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc06f610 0x7fb5fc0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 --2- 172.21.15.83:0/3475410510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc0af040 0x7fb5fc0cc5d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 -- 172.21.15.83:0/3475410510 >> 172.21.15.83:0/3475410510 conn(0x7fb5fc0b2c10 msgr2=0x7fb5fc0b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 -- 172.21.15.83:0/3475410510 shutdown_connections 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.806+0000 7fb600dd2700 1 -- 172.21.15.83:0/3475410510 wait complete. 2023-11-28T11:09:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 Processor -- start 2023-11-28T11:09:56.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 -- start start 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 0x7fb5fc0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5fc06f610 0x7fb5fc0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc0af040 0x7fb5fc0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0c66e0 con 0x7fb5fc06c500 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b1630 con 0x7fb5fc06f610 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.808+0000 7fb600dd2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0c3d30 con 0x7fb5fc0af040 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 0x7fb5fc0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 0x7fb5fc0c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50962/0 (socket says 172.21.15.83:50962) 2023-11-28T11:09:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 -- 172.21.15.83:0/2198551273 learned_addr learned my addr 172.21.15.83:0/2198551273 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5f9d9b700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5fc06f610 0x7fb5fc0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fad9d700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc0af040 0x7fb5fc0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 -- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc0af040 msgr2=0x7fb5fc0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc0af040 0x7fb5fc0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 -- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5fc06f610 msgr2=0x7fb5fc0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5fc06f610 0x7fb5fc0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 -- 172.21.15.83:0/2198551273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00094a0 con 0x7fb5fc06c500 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5fa59c700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 0x7fb5fc0c37f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0729d0 tx=0x7fb5f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00036a0 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f001f070 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc0ccb10 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0003a60 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.809+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc0ca420 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.810+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc0af410 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.811+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5f000f050 con 0x7fb5fc06c500 2023-11-28T11:09:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.811+0000 7fb5eaffd700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.811+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb5f0021070 con 0x7fb5fc06c500 2023-11-28T11:09:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.811+0000 7fb5f9d9b700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.811+0000 7fb5f9d9b700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc15bec0 tx=0x7fb5ec007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.815+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5f0053f30 con 0x7fb5fc06c500 2023-11-28T11:09:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.998+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5fc0742b0 con 0x7fb5fc06c500 2023-11-28T11:09:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:56.998+0000 7fb5eaffd700 1 -- 172.21.15.83:0/2198551273 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5f0015020 con 0x7fb5fc06c500 2023-11-28T11:09:56.999 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:09:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5dc0682f0 msgr2=0x7fb5dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc15bec0 tx=0x7fb5ec007e70 comp rx=0 tx=0).stop 2023-11-28T11:09:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 msgr2=0x7fb5fc0c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 0x7fb5fc0c37f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0729d0 tx=0x7fb5f0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 shutdown_connections 2023-11-28T11:09:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5fc0af040 0x7fb5fc0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5dc0682f0 0x7fb5dc06a7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc06c500 0x7fb5fc0c37f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 --2- 172.21.15.83:0/2198551273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5fc06f610 0x7fb5fc0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 >> 172.21.15.83:0/2198551273 conn(0x7fb5fc0b2c10 msgr2=0x7fb5fc0ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 shutdown_connections 2023-11-28T11:09:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.000+0000 7fb600dd2700 1 -- 172.21.15.83:0/2198551273 wait complete. 2023-11-28T11:09:57.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.135+0000 7f1b91639700 1 Processor -- start 2023-11-28T11:09:57.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.135+0000 7f1b91639700 1 -- start start 2023-11-28T11:09:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b91639700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b91639700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b8c0b4940 0x7f1b8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b91639700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c0b52d0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b91639700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8c0cd520 con 0x7f1b8c0b4940 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b91639700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0cf400 con 0x7f1b8c0b8570 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b8c0b4940 0x7f1b8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b8c0b4940 0x7f1b8c0b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50974/0 (socket says 172.21.15.83:50974) 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/3606036794 learned_addr learned my addr 172.21.15.83:0/3606036794 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8affd700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/3606036794 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 msgr2=0x7f1b8c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/3606036794 >> v1:172.21.15.90:6789/0 conn(0x7f1b8c0d2880 legacy=0x7f1b8c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/3606036794 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0cd7c0 con 0x7f1b8c0b4940 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8affd700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.136+0000 7f1b8b7fe700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c0b4d90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009e50 tx=0x7f1b8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa440bec0a1b09 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b89ffb700 1 -- 172.21.15.83:0/3606036794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400e070 con 0x7f1b8c0b4940 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b89ffb700 1 -- 172.21.15.83:0/3606036794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b84003a20 con 0x7f1b8c0b4940 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b89ffb700 1 -- 172.21.15.83:0/3606036794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84003dd0 con 0x7f1b8c0b4940 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 -- 172.21.15.83:0/3606036794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0b4940 msgr2=0x7f1b8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c0b4d90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009e50 tx=0x7f1b8400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 -- 172.21.15.83:0/3606036794 shutdown_connections 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c0b4d90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 --2- 172.21.15.83:0/3606036794 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 -- 172.21.15.83:0/3606036794 >> 172.21.15.83:0/3606036794 conn(0x7f1b8c0d30a0 msgr2=0x7f1b8c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 -- 172.21.15.83:0/3606036794 shutdown_connections 2023-11-28T11:09:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.137+0000 7f1b91639700 1 -- 172.21.15.83:0/3606036794 wait complete. 2023-11-28T11:09:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.138+0000 7f1b91639700 1 Processor -- start 2023-11-28T11:09:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 -- start start 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 0x7f1b8c1f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c14daa0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c14b1a0 con 0x7f1b8c0b4940 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b91639700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c14dcd0 con 0x7f1b8c0b8570 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b8b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 0x7f1b8c1f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b8b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 0x7f1b8c1f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50980/0 (socket says 172.21.15.83:50980) 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.139+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/522772314 learned_addr learned my addr 172.21.15.83:0/522772314 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8affd700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8a7fc700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 msgr2=0x7f1b8c14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8b7fe700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b8c0b4940 msgr2=0x7f1b8c14d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8b7fe700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c14d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8b7fe700 1 -- 172.21.15.83:0/522772314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b84009b30 con 0x7f1b8c0d2880 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8affd700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8a7fc700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b8b7fe700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 0x7f1b8c1f5e70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1b84015040 tx=0x7f1b84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84004440 con 0x7f1b8c0d2880 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b840045c0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c0cf6e0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84004970 con 0x7f1b8c0d2880 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.140+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c0cdaa0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.141+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8c0cf0f0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.141+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b84022030 con 0x7f1b8c0d2880 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.142+0000 7f1b737fe700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b780682f0 0x7f1b7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.142+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1b84087130 con 0x7f1b8c0d2880 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.142+0000 7f1b8affd700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b780682f0 0x7f1b7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.142+0000 7f1b8affd700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b780682f0 0x7f1b7806a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c1f3050 tx=0x7f1b80008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.146+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b840537d0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.328+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b8c0b52d0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.329+0000 7f1b737fe700 1 -- 172.21.15.83:0/522772314 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b8c0b52d0 con 0x7f1b8c0d2880 2023-11-28T11:09:57.330 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:09:57.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b780682f0 msgr2=0x7f1b7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b780682f0 0x7f1b7806a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c1f3050 tx=0x7f1b80008040 comp rx=0 tx=0).stop 2023-11-28T11:09:57.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 msgr2=0x7f1b8c1f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 0x7f1b8c1f5e70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1b84015040 tx=0x7f1b84004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 shutdown_connections 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b780682f0 0x7f1b7806a7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b8c0d2880 0x7f1b8c1f5e70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b8c0b4940 0x7f1b8c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.330+0000 7f1b91639700 1 --2- 172.21.15.83:0/522772314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8c0b8570 0x7f1b8c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.331+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 >> 172.21.15.83:0/522772314 conn(0x7f1b8c0d30a0 msgr2=0x7f1b8c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.331+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 shutdown_connections 2023-11-28T11:09:57.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.331+0000 7f1b91639700 1 -- 172.21.15.83:0/522772314 wait complete. 2023-11-28T11:09:57.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.465+0000 7f48879ba700 1 Processor -- start 2023-11-28T11:09:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.465+0000 7f48879ba700 1 -- start start 2023-11-28T11:09:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f48879ba700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f488015c890 0x7f48800aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f48879ba700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f488006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f48879ba700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f488006e1a0 0x7f48801604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f48879ba700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488006e620 con 0x7f488015c890 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f48879ba700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800b5020 con 0x7f488006e1a0 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f48879ba700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488006d4a0 con 0x7f48800b4bb0 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f488015c890 0x7f48800aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f488015c890 0x7f48800aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50984/0 (socket says 172.21.15.83:50984) 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 -- 172.21.15.83:0/4188519242 learned_addr learned my addr 172.21.15.83:0/4188519242 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4884f55700 1 --2- 172.21.15.83:0/4188519242 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f488006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885f57700 1 --2- 172.21.15.83:0/4188519242 >> v2:172.21.15.90:3300/0 conn(0x7f488006e1a0 0x7f48801604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 -- 172.21.15.83:0/4188519242 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 msgr2=0x7f488006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 --2- 172.21.15.83:0/4188519242 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f488006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 -- 172.21.15.83:0/4188519242 >> v2:172.21.15.90:3300/0 conn(0x7f488006e1a0 msgr2=0x7f48801604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.466+0000 7f4885756700 1 --2- 172.21.15.83:0/4188519242 >> v2:172.21.15.90:3300/0 conn(0x7f488006e1a0 0x7f48801604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f4885756700 1 -- 172.21.15.83:0/4188519242 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800b5ae0 con 0x7f488015c890 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f4885f57700 1 --2- 172.21.15.83:0/4188519242 >> v2:172.21.15.90:3300/0 conn(0x7f488006e1a0 0x7f48801604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f4885756700 1 --2- 172.21.15.83:0/4188519242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800aee60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4870006fd0 tx=0x7f4870009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ecb53581288432 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f4877fff700 1 -- 172.21.15.83:0/4188519242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870005e30 con 0x7f488015c890 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f4877fff700 1 -- 172.21.15.83:0/4188519242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870004030 con 0x7f488015c890 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f4877fff700 1 -- 172.21.15.83:0/4188519242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700043e0 con 0x7f488015c890 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 -- 172.21.15.83:0/4188519242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 msgr2=0x7f48800aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 --2- 172.21.15.83:0/4188519242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800aee60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4870006fd0 tx=0x7f4870009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 -- 172.21.15.83:0/4188519242 shutdown_connections 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 --2- 172.21.15.83:0/4188519242 >> v2:172.21.15.90:3300/0 conn(0x7f488006e1a0 0x7f48801604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 --2- 172.21.15.83:0/4188519242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800aee60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 --2- 172.21.15.83:0/4188519242 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f488006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.467+0000 7f48879ba700 1 -- 172.21.15.83:0/4188519242 >> 172.21.15.83:0/4188519242 conn(0x7f48800b2b60 msgr2=0x7f48800b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.468+0000 7f48879ba700 1 -- 172.21.15.83:0/4188519242 shutdown_connections 2023-11-28T11:09:57.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.468+0000 7f48879ba700 1 -- 172.21.15.83:0/4188519242 wait complete. 2023-11-28T11:09:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 Processor -- start 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 -- start start 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f488006e1a0 0x7f48801561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f48800c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4880156850 con 0x7f488015c890 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4880156b30 con 0x7f488006e1a0 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.469+0000 7f48879ba700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4880156e10 con 0x7f48800b4bb0 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50992/0 (socket says 172.21.15.83:50992) 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 -- 172.21.15.83:0/2848988756 learned_addr learned my addr 172.21.15.83:0/2848988756 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4884f55700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f48800c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885756700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f488006e1a0 0x7f48801561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 -- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 msgr2=0x7f48800c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f48800c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 -- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f488006e1a0 msgr2=0x7f48801561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f488006e1a0 0x7f48801561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 -- 172.21.15.83:0/2848988756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009500 con 0x7f488015c890 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4885f57700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800c3db0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f487800cd10 tx=0x7f487800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800f480 con 0x7f488015c890 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487800f600 con 0x7f488015c890 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48800b5240 con 0x7f488015c890 2023-11-28T11:09:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800f9b0 con 0x7f488015c890 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.470+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488006c400 con 0x7f488015c890 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.471+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48800b5520 con 0x7f488015c890 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.472+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4878009030 con 0x7f488015c890 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.472+0000 7f4875ffb700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48640682f0 0x7f486406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.472+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4878086710 con 0x7f488015c890 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.472+0000 7f4885756700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48640682f0 0x7f486406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.472+0000 7f4885756700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48640682f0 0x7f486406a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4870014040 tx=0x7f4870008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.476+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4878052dd0 con 0x7f488015c890 2023-11-28T11:09:57.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.657+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48800d05b0 con 0x7f488015c890 2023-11-28T11:09:57.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.658+0000 7f4875ffb700 1 -- 172.21.15.83:0/2848988756 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48780563f0 con 0x7f488015c890 2023-11-28T11:09:57.658 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:09:57.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48640682f0 msgr2=0x7f486406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48640682f0 0x7f486406a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4870014040 tx=0x7f4870008070 comp rx=0 tx=0).stop 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 msgr2=0x7f48800c3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800c3db0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f487800cd10 tx=0x7f487800a560 comp rx=0 tx=0).stop 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 shutdown_connections 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48640682f0 0x7f486406a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c890 0x7f48800c3db0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f488006e1a0 0x7f48801561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 --2- 172.21.15.83:0/2848988756 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48800b4bb0 0x7f48800c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 >> 172.21.15.83:0/2848988756 conn(0x7f48800b2b60 msgr2=0x7f48800aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 shutdown_connections 2023-11-28T11:09:57.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.660+0000 7f48879ba700 1 -- 172.21.15.83:0/2848988756 wait complete. 2023-11-28T11:09:57.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.792+0000 7fd53dd15700 1 Processor -- start 2023-11-28T11:09:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.792+0000 7fd53dd15700 1 -- start start 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd53dd15700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd5380ce1b0 0x7fd53815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd53dd15700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd538151a30 con 0x7fd5380d0ae0 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd53dd15700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5380cb460 con 0x7fd53815e5c0 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd53dd15700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5380cb740 con 0x7fd5380ce1b0 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd536ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd5380ce1b0 0x7fd53815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd537fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd5380d0ae0 0x7fd5380d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48862/0 (socket says 172.21.15.83:48862) 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd537fff700 1 -- 172.21.15.83:0/3513508417 learned_addr learned my addr 172.21.15.83:0/3513508417 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd536ffd700 1 -- 172.21.15.83:0/3513508417 >> v1:172.21.15.90:6789/0 conn(0x7fd53815e5c0 legacy=0x7fd53815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd536ffd700 1 -- 172.21.15.83:0/3513508417 >> v1:172.21.15.83:6789/0 conn(0x7fd5380d0ae0 legacy=0x7fd5380d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd536ffd700 1 -- 172.21.15.83:0/3513508417 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53815abb0 con 0x7fd5380ce1b0 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd5367fc700 1 -- 172.21.15.83:0/3513508417 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493217865 0 0) 0x7fd538151a30 con 0x7fd5380d0ae0 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd536ffd700 1 --2- 172.21.15.83:0/3513508417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380ce1b0 0x7fd53815b3b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd528009840 tx=0x7fd528009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b93239423bf5f7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd5367fc700 1 -- 172.21.15.83:0/3513508417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011830 con 0x7fd5380ce1b0 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd5367fc700 1 -- 172.21.15.83:0/3513508417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528011990 con 0x7fd5380ce1b0 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.793+0000 7fd5367fc700 1 -- 172.21.15.83:0/3513508417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011d20 con 0x7fd5380ce1b0 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 -- 172.21.15.83:0/3513508417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380ce1b0 msgr2=0x7fd53815b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 --2- 172.21.15.83:0/3513508417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380ce1b0 0x7fd53815b3b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd528009840 tx=0x7fd528009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 -- 172.21.15.83:0/3513508417 shutdown_connections 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 --2- 172.21.15.83:0/3513508417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380ce1b0 0x7fd53815b3b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 -- 172.21.15.83:0/3513508417 >> 172.21.15.83:0/3513508417 conn(0x7fd53806ea50 msgr2=0x7fd538070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 -- 172.21.15.83:0/3513508417 shutdown_connections 2023-11-28T11:09:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.794+0000 7fd53dd15700 1 -- 172.21.15.83:0/3513508417 wait complete. 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.795+0000 7fd53dd15700 1 Processor -- start 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.795+0000 7fd53dd15700 1 -- start start 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd53dd15700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5380d0ae0 0x7fd5380c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd53dd15700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd53815e5c0 0x7fd5380b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd53dd15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 0x7fd5380b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd53dd15700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c6da0 con 0x7fd5380b4aa0 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd53dd15700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c6120 con 0x7fd5380d0ae0 2023-11-28T11:09:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd53dd15700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c6400 con 0x7fd53815e5c0 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd537fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 0x7fd5380b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd537fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 0x7fd5380b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51008/0 (socket says 172.21.15.83:51008) 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd537fff700 1 -- 172.21.15.83:0/2519017446 learned_addr learned my addr 172.21.15.83:0/2519017446 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd5377fe700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5380d0ae0 0x7fd5380c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.796+0000 7fd536ffd700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd53815e5c0 0x7fd5380b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd537fff700 1 -- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd53815e5c0 msgr2=0x7fd5380b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd537fff700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd53815e5c0 0x7fd5380b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd537fff700 1 -- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5380d0ae0 msgr2=0x7fd5380c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd537fff700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5380d0ae0 0x7fd5380c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd537fff700 1 -- 172.21.15.83:0/2519017446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280094a0 con 0x7fd5380b4aa0 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd537fff700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 0x7fd5380b6120 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd530002230 tx=0x7fd53000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000aac0 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000ac40 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300070d0 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53815b8f0 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.797+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53815ae90 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.798+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53815ba80 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.798+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd530010070 con 0x7fd5380b4aa0 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.799+0000 7fd52ffff700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.799+0000 7fd5377fe700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.799+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5300856a0 con 0x7fd5380b4aa0 2023-11-28T11:09:57.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.799+0000 7fd5377fe700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd520007fd0 tx=0x7fd520009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:57.803+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd530051dc0 con 0x7fd5380b4aa0 2023-11-28T11:09:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.004+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5380c6ee0 con 0x7fd5380b4aa0 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.004+0000 7fd52ffff700 1 -- 172.21.15.83:0/2519017446 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd5380c6ee0 con 0x7fd5380b4aa0 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 msgr2=0x7fd51806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd520007fd0 tx=0x7fd520009040 comp rx=0 tx=0).stop 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 msgr2=0x7fd5380b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 0x7fd5380b6120 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd530002230 tx=0x7fd53000cf50 comp rx=0 tx=0).stop 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 shutdown_connections 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd5180682f0 0x7fd51806a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5380b4aa0 0x7fd5380b6120 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5380d0ae0 0x7fd5380c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 --2- 172.21.15.83:0/2519017446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd53815e5c0 0x7fd5380b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 >> 172.21.15.83:0/2519017446 conn(0x7fd5380ce1b0 msgr2=0x7fd53815c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.006+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 shutdown_connections 2023-11-28T11:09:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.007+0000 7fd53dd15700 1 -- 172.21.15.83:0/2519017446 wait complete. 2023-11-28T11:09:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:09:58.019 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:09:58.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:58.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:58.087 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:09:58.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:58.087 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:09:58.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.272+0000 7f66d79c4700 1 Processor -- start 2023-11-28T11:09:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.273+0000 7f66d79c4700 1 -- start start 2023-11-28T11:09:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.273+0000 7f66d79c4700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f66d0159830 0x7f66d00cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d79c4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d006c5f0 0x7f66d015de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d79c4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66d00b18a0 0x7f66d00b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d79c4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d015d480 con 0x7f66d00b18a0 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d79c4700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d00b2260 con 0x7f66d0159830 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d79c4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d006ca60 con 0x7f66d006c5f0 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66d00b18a0 0x7f66d00b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66d00b18a0 0x7f66d00b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51016/0 (socket says 172.21.15.83:51016) 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 -- 172.21.15.83:0/2102649155 learned_addr learned my addr 172.21.15.83:0/2102649155 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d4f5f700 1 --2- 172.21.15.83:0/2102649155 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d006c5f0 0x7f66d015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 -- 172.21.15.83:0/2102649155 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d006c5f0 msgr2=0x7f66d015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 --2- 172.21.15.83:0/2102649155 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d006c5f0 0x7f66d015de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5760700 1 --2- 172.21.15.83:0/2102649155 >> v2:172.21.15.90:3300/0 conn(0x7f66d0159830 0x7f66d00cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 -- 172.21.15.83:0/2102649155 >> v2:172.21.15.90:3300/0 conn(0x7f66d0159830 msgr2=0x7f66d00cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 --2- 172.21.15.83:0/2102649155 >> v2:172.21.15.90:3300/0 conn(0x7f66d0159830 0x7f66d00cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 -- 172.21.15.83:0/2102649155 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d015e3d0 con 0x7f66d00b18a0 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66d5f61700 1 --2- 172.21.15.83:0/2102649155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d00b18a0 0x7f66d00b1d20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f66d006dcb0 tx=0x7f66c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cac005a4df0876 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66c7fff700 1 -- 172.21.15.83:0/2102649155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c800e070 con 0x7f66d00b18a0 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66c7fff700 1 -- 172.21.15.83:0/2102649155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c800b9c0 con 0x7f66d00b18a0 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.274+0000 7f66c7fff700 1 -- 172.21.15.83:0/2102649155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c800bd70 con 0x7f66d00b18a0 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 -- 172.21.15.83:0/2102649155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d00b18a0 msgr2=0x7f66d00b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2102649155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d00b18a0 0x7f66d00b1d20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f66d006dcb0 tx=0x7f66c800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 -- 172.21.15.83:0/2102649155 shutdown_connections 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2102649155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d00b18a0 0x7f66d00b1d20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2102649155 >> v2:172.21.15.90:3300/0 conn(0x7f66d0159830 0x7f66d00cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2102649155 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d006c5f0 0x7f66d015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 -- 172.21.15.83:0/2102649155 >> 172.21.15.83:0/2102649155 conn(0x7f66d006e9a0 msgr2=0x7f66d00cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 -- 172.21.15.83:0/2102649155 shutdown_connections 2023-11-28T11:09:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.275+0000 7f66d79c4700 1 -- 172.21.15.83:0/2102649155 wait complete. 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.276+0000 7f66d79c4700 1 Processor -- start 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.276+0000 7f66d79c4700 1 -- start start 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d79c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 0x7f66d00c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d79c4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d00b18a0 0x7f66d01f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d79c4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d0159830 0x7f66d0153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d79c4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01eff80 con 0x7f66d006c5f0 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d79c4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00c88e0 con 0x7f66d00b18a0 2023-11-28T11:09:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d79c4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01f3cf0 con 0x7f66d0159830 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 0x7f66d00c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 0x7f66d00c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51018/0 (socket says 172.21.15.83:51018) 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 -- 172.21.15.83:0/2357165401 learned_addr learned my addr 172.21.15.83:0/2357165401 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5f61700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d0159830 0x7f66d0153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d4f5f700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d00b18a0 0x7f66d01f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 -- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d0159830 msgr2=0x7f66d0153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d0159830 0x7f66d0153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 -- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d00b18a0 msgr2=0x7f66d01f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d00b18a0 0x7f66d01f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.277+0000 7f66d5760700 1 -- 172.21.15.83:0/2357165401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8009b30 con 0x7f66d006c5f0 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.278+0000 7f66d5760700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 0x7f66d00c6520 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f66c000c7f0 tx=0x7f66c000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.278+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0011800 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.278+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c0011960 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.278+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d00cfa70 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.278+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0011cd0 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.278+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d00cf790 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.279+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d015d690 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.279+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66c0017030 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.279+0000 7f66c5ffb700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b40682f0 0x7f66b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.279+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f66c0085730 con 0x7f66d006c5f0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.280+0000 7f66d4f5f700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b40682f0 0x7f66b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.280+0000 7f66d4f5f700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b40682f0 0x7f66b406a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66d00ca3d0 tx=0x7f66cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.284+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66c0051e50 con 0x7f66d006c5f0 2023-11-28T11:09:58.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.511+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66d0074320 con 0x7f66d006c5f0 2023-11-28T11:09:58.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.511+0000 7f66c5ffb700 1 -- 172.21.15.83:0/2357165401 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f66c0055470 con 0x7f66d006c5f0 2023-11-28T11:09:58.512 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:58.512 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b40682f0 msgr2=0x7f66b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b40682f0 0x7f66b406a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66d00ca3d0 tx=0x7f66cc008040 comp rx=0 tx=0).stop 2023-11-28T11:09:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 msgr2=0x7f66d00c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 0x7f66d00c6520 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f66c000c7f0 tx=0x7f66c000cb00 comp rx=0 tx=0).stop 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 shutdown_connections 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66d0159830 0x7f66d0153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b40682f0 0x7f66b406a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66d006c5f0 0x7f66d00c6520 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 --2- 172.21.15.83:0/2357165401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d00b18a0 0x7f66d01f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 >> 172.21.15.83:0/2357165401 conn(0x7f66d006e9a0 msgr2=0x7f66d01e32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 shutdown_connections 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.513+0000 7f66d79c4700 1 -- 172.21.15.83:0/2357165401 wait complete. 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:09:58.525 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:09:58.690 INFO:teuthology.orchestra.run.smithi090.stdout:{"message":"","return_code":0} 2023-11-28T11:09:58.690 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:09:58.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:58.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.811+0000 7f5ddbcfd700 1 Processor -- start 2023-11-28T11:09:58.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.811+0000 7f5ddbcfd700 1 -- start start 2023-11-28T11:09:58.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5ddbcfd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5ddbcfd700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5dd4071ae0 0x7f5dd40c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5ddbcfd700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40cac90 con 0x7f5dd40c9de0 2023-11-28T11:09:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5ddbcfd700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40ca1f0 con 0x7f5dd4071ae0 2023-11-28T11:09:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5ddbcfd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40ca3f0 con 0x7f5dd4071710 2023-11-28T11:09:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5dd9298700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5dd4071ae0 0x7f5dd40c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5dd9a99700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5dd9298700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5dd4071ae0 0x7f5dd40c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:38020/0 (socket says 172.21.15.83:38020) 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5dd9298700 1 -- 172.21.15.83:0/685180548 learned_addr learned my addr 172.21.15.83:0/685180548 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.812+0000 7f5dda29a700 1 --1- 172.21.15.83:0/685180548 >> v1:172.21.15.90:6789/0 conn(0x7f5dd40c9de0 0x7f5dd41586b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:47236/0 (socket says 172.21.15.83:47236) 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd9298700 1 -- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 msgr2=0x7f5dd40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd9298700 1 --2- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd40cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd9298700 1 -- 172.21.15.83:0/685180548 >> v1:172.21.15.90:6789/0 conn(0x7f5dd40c9de0 legacy=0x7f5dd41586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd9298700 1 -- 172.21.15.83:0/685180548 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40ca6d0 con 0x7f5dd4071ae0 2023-11-28T11:09:58.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd8a97700 1 -- 172.21.15.83:0/685180548 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324482240 0 0) 0x7f5dd40cac90 con 0x7f5dd40c9de0 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd9a99700 1 --2- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd9298700 1 --2- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c98a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098f0 tx=0x7f5dd0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd7c86c77f0d9ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd8a97700 1 -- 172.21.15.83:0/685180548 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0011830 con 0x7f5dd4071ae0 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd8a97700 1 -- 172.21.15.83:0/685180548 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0011990 con 0x7f5dd4071ae0 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5dd8a97700 1 -- 172.21.15.83:0/685180548 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0011d20 con 0x7f5dd4071ae0 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 msgr2=0x7f5dd40c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c98a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098f0 tx=0x7f5dd0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/685180548 shutdown_connections 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/685180548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c98a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.813+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/685180548 >> 172.21.15.83:0/685180548 conn(0x7f5dd40cec90 msgr2=0x7f5dd40cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.814+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/685180548 shutdown_connections 2023-11-28T11:09:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.814+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/685180548 wait complete. 2023-11-28T11:09:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 Processor -- start 2023-11-28T11:09:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 -- start start 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd4157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 0x7f5dd40bc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c80b0 con 0x7f5dd40c9de0 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.815+0000 7f5ddbcfd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c8390 con 0x7f5dd4071ae0 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5ddbcfd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c8670 con 0x7f5dd4071710 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 0x7f5dd40bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 0x7f5dd40bc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51036/0 (socket says 172.21.15.83:51036) 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 -- 172.21.15.83:0/2003304886 learned_addr learned my addr 172.21.15.83:0/2003304886 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dd9a99700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd4157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dd9298700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 -- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 msgr2=0x7f5dd4157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd4157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 -- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 msgr2=0x7f5dd40c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 -- 172.21.15.83:0/2003304886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40094a0 con 0x7f5dd40c9de0 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dda29a700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 0x7f5dd40bc390 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc002230 tx=0x7f5dcc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00ac20 con 0x7f5dd40c9de0 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc00ada0 con 0x7f5dd40c9de0 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc007310 con 0x7f5dd40c9de0 2023-11-28T11:09:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd0009550 con 0x7f5dd40c9de0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.816+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd406dae0 con 0x7f5dd40c9de0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.817+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd40ca9b0 con 0x7f5dd40c9de0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.818+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5dcc010070 con 0x7f5dd40c9de0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.818+0000 7f5dca7fc700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5db80682f0 0x7f5db806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.818+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5dcc085800 con 0x7f5dd40c9de0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.818+0000 7f5dd9a99700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5db80682f0 0x7f5db806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.818+0000 7f5dd9a99700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5db80682f0 0x7f5db806a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009c60 tx=0x7f5dc4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:58.822+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5dcc051f20 con 0x7f5dd40c9de0 2023-11-28T11:09:59.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.049+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dd40c24a0 con 0x7f5dd40c9de0 2023-11-28T11:09:59.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.049+0000 7f5dca7fc700 1 -- 172.21.15.83:0/2003304886 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f5dcc055540 con 0x7f5dd40c9de0 2023-11-28T11:09:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:09:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:59.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5db80682f0 msgr2=0x7f5db806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5db80682f0 0x7f5db806a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009c60 tx=0x7f5dc4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:59.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 msgr2=0x7f5dd40bc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 0x7f5dd40bc390 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc002230 tx=0x7f5dcc00a060 comp rx=0 tx=0).stop 2023-11-28T11:09:59.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 shutdown_connections 2023-11-28T11:09:59.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5db80682f0 0x7f5db806a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd40c9de0 0x7f5dd40bc390 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5dd4071710 0x7f5dd4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 --2- 172.21.15.83:0/2003304886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dd4071ae0 0x7f5dd40c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 >> 172.21.15.83:0/2003304886 conn(0x7f5dd40cec90 msgr2=0x7f5dd40cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 shutdown_connections 2023-11-28T11:09:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:09:59.051+0000 7f5ddbcfd700 1 -- 172.21.15.83:0/2003304886 wait complete. 2023-11-28T11:09:59.054 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:09:59.064 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parentdir mytag 2023-11-28T11:09:59.208 INFO:teuthology.orchestra.run.smithi090.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T11:09:59.208 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T11:10:09.209 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.262+0000 7f6df27af080 1 Processor -- start 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.262+0000 7f6df27af080 1 -- start start 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6df27af080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 0x55f30ffa87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6df27af080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30ff03bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6df27af080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ffb1300 0x55f30ff235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6df27af080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30fe13fe0 con 0x55f30ff09030 2023-11-28T11:10:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6df27af080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ff04110 con 0x55f30ffb0f30 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6df27af080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f30ffa7e20 con 0x55f30ffb1300 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6de88b2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 0x55f30ffa87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6de88b2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 0x55f30ffa87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51046/0 (socket says 172.21.15.83:51046) 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6de88b2700 1 -- 172.21.15.83:0/4133440158 learned_addr learned my addr 172.21.15.83:0/4133440158 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6de90b3700 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ffb1300 0x55f30ff235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.263+0000 7f6de3fff700 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30ff03bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de88b2700 1 -- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ffb1300 msgr2=0x55f30ff235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de88b2700 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ffb1300 0x55f30ff235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de88b2700 1 -- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 msgr2=0x55f30ff03bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de88b2700 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30ff03bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de88b2700 1 -- 172.21.15.83:0/4133440158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc005d20 con 0x55f30ff09030 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de88b2700 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 0x55f30ffa87c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1224378fc6e713e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de2ffd700 1 -- 172.21.15.83:0/4133440158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc008b60 con 0x55f30ff09030 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de2ffd700 1 -- 172.21.15.83:0/4133440158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc008ce0 con 0x55f30ff09030 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.264+0000 7f6de2ffd700 1 -- 172.21.15.83:0/4133440158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01e690 con 0x55f30ff09030 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 -- 172.21.15.83:0/4133440158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 msgr2=0x55f30ffa87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 0x55f30ffa87c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc00d000 comp rx=0 tx=0).stop 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 -- 172.21.15.83:0/4133440158 shutdown_connections 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ffb1300 0x55f30ff235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ff09030 0x55f30ffa87c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 --2- 172.21.15.83:0/4133440158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30ff03bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 -- 172.21.15.83:0/4133440158 >> 172.21.15.83:0/4133440158 conn(0x55f30ff25370 msgr2=0x55f30ffaffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 -- 172.21.15.83:0/4133440158 shutdown_connections 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 -- 172.21.15.83:0/4133440158 wait complete. 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 Processor -- start 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.265+0000 7f6df27af080 1 -- start start 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6df27af080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ff09030 0x55f30ffa64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6df27af080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30feda270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6df27af080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ffb1300 0x55f30feda7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6df27af080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ffa8970 con 0x55f30ffb1300 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6df27af080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ffa8f30 con 0x55f30ffb0f30 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6df27af080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f31004aad0 con 0x55f30ff09030 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6de90b3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ffb1300 0x55f30feda7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6de90b3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ffb1300 0x55f30feda7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51056/0 (socket says 172.21.15.83:51056) 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6de90b3700 1 -- 172.21.15.83:0/2726861629 learned_addr learned my addr 172.21.15.83:0/2726861629 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6de88b2700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ff09030 0x55f30ffa64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.266+0000 7f6de3fff700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30feda270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de90b3700 1 -- 172.21.15.83:0/2726861629 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ff09030 msgr2=0x55f30ffa64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de90b3700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f30ff09030 0x55f30ffa64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de90b3700 1 -- 172.21.15.83:0/2726861629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 msgr2=0x55f30feda270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de90b3700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f30ffb0f30 0x55f30feda270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de90b3700 1 -- 172.21.15.83:0/2726861629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f30fe464f0 con 0x55f30ffb1300 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de90b3700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ffb1300 0x55f30feda7b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6de4004e70 tx=0x7f6de40017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de1ffb700 1 -- 172.21.15.83:0/2726861629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4001fe0 con 0x55f30ffb1300 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de1ffb700 1 -- 172.21.15.83:0/2726861629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4011380 con 0x55f30ffb1300 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f30fe2c120 con 0x55f30ffb1300 2023-11-28T11:10:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6de1ffb700 1 -- 172.21.15.83:0/2726861629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4011730 con 0x55f30ffb1300 2023-11-28T11:10:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.267+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f30fe1f830 con 0x55f30ffb1300 2023-11-28T11:10:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.268+0000 7f6de1ffb700 1 -- 172.21.15.83:0/2726861629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6de401a030 con 0x55f30ffb1300 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6de1ffb700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6dcc066c10 0x7f6dcc0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6de1ffb700 1 -- 172.21.15.83:0/2726861629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6de4083bf0 con 0x55f30ffb1300 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6de88b2700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6dcc066c10 0x7f6dcc0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6df27af080 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x55f30ffa93c0 0x55f30ffab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 --> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x55f30ff07c20 con 0x55f30ffa93c0 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6de3fff700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x55f30ffa93c0 0x55f30ffab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.269+0000 7f6de88b2700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6dcc066c10 0x7f6dcc0690c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55f30ffafaa0 tx=0x7f6ddc014040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6de3fff700 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x55f30ffa93c0 0x55f30ffab040 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6de3fff700 1 -- 172.21.15.83:0/2726861629 <== osd.0 v2:172.21.15.83:6800/103534 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f6dd4009790 con 0x55f30ffa93c0 2023-11-28T11:10:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x55f30ffa93c0 msgr2=0x55f30ffab040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x55f30ffa93c0 0x55f30ffab040 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6dcc066c10 msgr2=0x7f6dcc0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6dcc066c10 0x7f6dcc0690c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55f30ffafaa0 tx=0x7f6ddc014040 comp rx=0 tx=0).stop 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ffb1300 msgr2=0x55f30feda7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 --2- 172.21.15.83:0/2726861629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30ffb1300 0x55f30feda7b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6de4004e70 tx=0x7f6de40017b0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6de90b3700 1 -- 172.21.15.83:0/2726861629 reap_dead start 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 shutdown_connections 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 >> 172.21.15.83:0/2726861629 conn(0x55f30ff25370 msgr2=0x55f30ffafd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 shutdown_connections 2023-11-28T11:10:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.270+0000 7f6df27af080 1 -- 172.21.15.83:0/2726861629 wait complete. 2023-11-28T11:10:09.278 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2023-11-28T11:10:09.278 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T11:10:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.323+0000 7f558d256080 1 Processor -- start 2023-11-28T11:10:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.323+0000 7f558d256080 1 -- start start 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f558d256080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30ddde0 0x55c3a30d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f558d256080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30de1b0 0x55c3a30e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f558d256080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 0x55c3a30dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f558d256080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a2f40fe0 con 0x55c3a30d4ce0 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f558d256080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3004be0 con 0x55c3a30de1b0 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f558d256080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3006810 con 0x55c3a30ddde0 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 0x55c3a30dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 0x55c3a30dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51070/0 (socket says 172.21.15.83:51070) 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 -- 172.21.15.83:0/3093411672 learned_addr learned my addr 172.21.15.83:0/3093411672 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583359700 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30ddde0 0x55c3a30d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5582b58700 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30de1b0 0x55c3a30e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 -- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30ddde0 msgr2=0x55c3a30d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30ddde0 0x55c3a30d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 -- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30de1b0 msgr2=0x55c3a30e0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30de1b0 0x55c3a30e0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.324+0000 7f5583b5a700 1 -- 172.21.15.83:0/3093411672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574005de0 con 0x55c3a30d4ce0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f5583b5a700 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 0x55c3a30dc7c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f557c006e70 tx=0x7f557c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b39684e7ef9123d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f5581b56700 1 -- 172.21.15.83:0/3093411672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c006310 con 0x55c3a30d4ce0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f5581b56700 1 -- 172.21.15.83:0/3093411672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c013420 con 0x55c3a30d4ce0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f5581b56700 1 -- 172.21.15.83:0/3093411672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c0137d0 con 0x55c3a30d4ce0 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 -- 172.21.15.83:0/3093411672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 msgr2=0x55c3a30dc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 0x55c3a30dc7c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f557c006e70 tx=0x7f557c001930 comp rx=0 tx=0).stop 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 -- 172.21.15.83:0/3093411672 shutdown_connections 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30d4ce0 0x55c3a30dc7c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30ddde0 0x55c3a30d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 --2- 172.21.15.83:0/3093411672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30de1b0 0x55c3a30e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 -- 172.21.15.83:0/3093411672 >> 172.21.15.83:0/3093411672 conn(0x55c3a3033fb0 msgr2=0x55c3a3031ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 -- 172.21.15.83:0/3093411672 shutdown_connections 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.325+0000 7f558d256080 1 -- 172.21.15.83:0/3093411672 wait complete. 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.326+0000 7f558d256080 1 Processor -- start 2023-11-28T11:10:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.326+0000 7f558d256080 1 -- start start 2023-11-28T11:10:09.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f558d256080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30d4ce0 0x55c3a3032950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f558d256080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30ddde0 0x55c3a304a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f558d256080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30de1b0 0x55c3a304aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f558d256080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a30dcb80 con 0x55c3a30de1b0 2023-11-28T11:10:09.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f558d256080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a30dc850 con 0x55c3a30ddde0 2023-11-28T11:10:09.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f558d256080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3a30cfc10 con 0x55c3a30d4ce0 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f5583b5a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30de1b0 0x55c3a304aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f5583b5a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30de1b0 0x55c3a304aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51076/0 (socket says 172.21.15.83:51076) 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f5583b5a700 1 -- 172.21.15.83:0/3371129530 learned_addr learned my addr 172.21.15.83:0/3371129530 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.327+0000 7f5583359700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30d4ce0 0x55c3a3032950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5582b58700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30ddde0 0x55c3a304a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583b5a700 1 -- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30d4ce0 msgr2=0x55c3a3032950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583b5a700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30d4ce0 0x55c3a3032950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583b5a700 1 -- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30ddde0 msgr2=0x55c3a304a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583b5a700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3a30ddde0 0x55c3a304a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583b5a700 1 -- 172.21.15.83:0/3371129530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c001e10 con 0x55c3a30de1b0 2023-11-28T11:10:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583359700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55c3a30d4ce0 0x55c3a3032950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5583b5a700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30de1b0 0x55c3a304aee0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f557c007fd0 tx=0x7f557c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5580b54700 1 -- 172.21.15.83:0/3371129530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c003650 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5580b54700 1 -- 172.21.15.83:0/3371129530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c0037b0 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f5580b54700 1 -- 172.21.15.83:0/3371129530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c003b60 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c3a2f59120 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.328+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3a2f4c830 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.329+0000 7f5580b54700 1 -- 172.21.15.83:0/3371129530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f557c023020 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f5580b54700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5570066c10 0x7f55700690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f5580b54700 1 -- 172.21.15.83:0/3371129530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f557c0860b0 con 0x55c3a30de1b0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f5583359700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5570066c10 0x7f55700690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f558d256080 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55c3a3032e90 0x55c3a30d3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- osd_op(unknown.0.0:1 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x55c3a30d01e0 con 0x55c3a3032e90 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f5583359700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5570066c10 0x7f55700690c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c3a3030720 tx=0x7f557800c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.330+0000 7f5582b58700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55c3a3032e90 0x55c3a30d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f5582b58700 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55c3a3032e90 0x55c3a30d3560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f5582b58700 1 -- 172.21.15.83:0/3371129530 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f5574019040 con 0x55c3a3032e90 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55c3a3032e90 msgr2=0x55c3a30d3560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55c3a3032e90 0x55c3a30d3560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5570066c10 msgr2=0x7f55700690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5570066c10 0x7f55700690c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c3a3030720 tx=0x7f557800c040 comp rx=0 tx=0).stop 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30de1b0 msgr2=0x55c3a304aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 --2- 172.21.15.83:0/3371129530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3a30de1b0 0x55c3a304aee0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f557c007fd0 tx=0x7f557c001250 comp rx=0 tx=0).stop 2023-11-28T11:10:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f5583b5a700 1 -- 172.21.15.83:0/3371129530 reap_dead start 2023-11-28T11:10:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 shutdown_connections 2023-11-28T11:10:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 >> 172.21.15.83:0/3371129530 conn(0x55c3a3033fb0 msgr2=0x55c3a30d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 shutdown_connections 2023-11-28T11:10:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.331+0000 7f558d256080 1 -- 172.21.15.83:0/3371129530 wait complete. 2023-11-28T11:10:09.339 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2023-11-28T11:10:09.339 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T11:10:09.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.381+0000 7f7cf380f080 1 Processor -- start 2023-11-28T11:10:09.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.381+0000 7f7cf380f080 1 -- start start 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cf380f080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acd780 0x561c07b56370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cf380f080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acf380 0x561c079f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cf380f080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 0x561c07a21af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cf380f080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c07930fe0 con 0x561c07acf750 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cf380f080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c07abd280 con 0x561c07acd780 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cf380f080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c07b55be0 con 0x561c07acf380 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cea113700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 0x561c07a21af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cea113700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 0x561c07a21af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51092/0 (socket says 172.21.15.83:51092) 2023-11-28T11:10:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7cea113700 1 -- 172.21.15.83:0/3064371134 learned_addr learned my addr 172.21.15.83:0/3064371134 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7ce9111700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acf380 0x561c079f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.382+0000 7f7ce9912700 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acd780 0x561c07b56370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cea113700 1 -- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acf380 msgr2=0x561c079f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cea113700 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acf380 0x561c079f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cea113700 1 -- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acd780 msgr2=0x561c07b56370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cea113700 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acd780 0x561c07b56370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cea113700 1 -- 172.21.15.83:0/3064371134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc005d20 con 0x561c07acf750 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7ce9111700 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acf380 0x561c079f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7ce9912700 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acd780 0x561c07b56370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cea113700 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 0x561c07a21af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006f80 tx=0x7f7ce4001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba4e2591d7f0172 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cdbfff700 1 -- 172.21.15.83:0/3064371134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4012600 con 0x561c07acf750 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cdbfff700 1 -- 172.21.15.83:0/3064371134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4012760 con 0x561c07acf750 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cdbfff700 1 -- 172.21.15.83:0/3064371134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4012b10 con 0x561c07acf750 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cf380f080 1 -- 172.21.15.83:0/3064371134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 msgr2=0x561c07a21af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.383+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 0x561c07a21af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006f80 tx=0x7f7ce4001820 comp rx=0 tx=0).stop 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 -- 172.21.15.83:0/3064371134 shutdown_connections 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf750 0x561c07a21af0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acd780 0x561c07b56370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3064371134 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acf380 0x561c079f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 -- 172.21.15.83:0/3064371134 >> 172.21.15.83:0/3064371134 conn(0x561c07a41af0 msgr2=0x561c07ad03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 -- 172.21.15.83:0/3064371134 shutdown_connections 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 -- 172.21.15.83:0/3064371134 wait complete. 2023-11-28T11:10:09.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 Processor -- start 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.384+0000 7f7cf380f080 1 -- start start 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7cf380f080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acd780 0x561c07ab8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7cf380f080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf380 0x561c07a17f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7cf380f080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acf750 0x561c07a18490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7cf380f080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c07ab9180 con 0x561c07acf380 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7cf380f080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c07a3e680 con 0x561c07acf750 2023-11-28T11:10:09.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7cf380f080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c07acbb90 con 0x561c07acd780 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7ce9111700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf380 0x561c07a17f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7ce9111700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf380 0x561c07a17f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51096/0 (socket says 172.21.15.83:51096) 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.385+0000 7f7ce9111700 1 -- 172.21.15.83:0/3009143748 learned_addr learned my addr 172.21.15.83:0/3009143748 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9912700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acd780 0x561c07ab8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7cea113700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acf750 0x561c07a18490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9111700 1 -- 172.21.15.83:0/3009143748 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acd780 msgr2=0x561c07ab8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9111700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561c07acd780 0x561c07ab8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9111700 1 -- 172.21.15.83:0/3009143748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acf750 msgr2=0x561c07a18490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9111700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c07acf750 0x561c07a18490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9111700 1 -- 172.21.15.83:0/3009143748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce0009450 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7ce9111700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf380 0x561c07a17f50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x561c079e49c0 tx=0x7f7cdc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7cdaffd700 1 -- 172.21.15.83:0/3009143748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc008690 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7cdaffd700 1 -- 172.21.15.83:0/3009143748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc009820 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c079634f0 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7cdaffd700 1 -- 172.21.15.83:0/3009143748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc009bb0 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.386+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c0793c4a0 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.387+0000 7f7cdaffd700 1 -- 172.21.15.83:0/3009143748 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7cdc009d10 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7cdaffd700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ccc066c10 0x7f7ccc0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7cdaffd700 1 -- 172.21.15.83:0/3009143748 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7cdc086970 con 0x561c07acf380 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7ce9912700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ccc066c10 0x7f7ccc0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x561c07ab93f0 0x561c07a258d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 --> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] -- osd_op(unknown.0.0:1 7.36 7:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x561c07a3ebd0 con 0x561c07ab93f0 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7cea113700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x561c07ab93f0 0x561c07a258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7ce9912700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ccc066c10 0x7f7ccc0690c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005fd0 tx=0x7f7ce000c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.388+0000 7f7cea113700 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x561c07ab93f0 0x561c07a258d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cea113700 1 -- 172.21.15.83:0/3009143748 <== osd.0 v2:172.21.15.83:6800/103534 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f7ce4012480 con 0x561c07ab93f0 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x561c07ab93f0 msgr2=0x561c07a258d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x561c07ab93f0 0x561c07a258d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ccc066c10 msgr2=0x7f7ccc0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7ccc066c10 0x7f7ccc0690c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005fd0 tx=0x7f7ce000c040 comp rx=0 tx=0).stop 2023-11-28T11:10:09.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf380 msgr2=0x561c07a17f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 --2- 172.21.15.83:0/3009143748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c07acf380 0x561c07a17f50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x561c079e49c0 tx=0x7f7cdc002fb0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cea113700 1 -- 172.21.15.83:0/3009143748 reap_dead start 2023-11-28T11:10:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 shutdown_connections 2023-11-28T11:10:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 >> 172.21.15.83:0/3009143748 conn(0x561c07a41af0 msgr2=0x561c07abb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 shutdown_connections 2023-11-28T11:10:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.389+0000 7f7cf380f080 1 -- 172.21.15.83:0/3009143748 wait complete. 2023-11-28T11:10:09.399 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2023-11-28T11:10:09.400 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T11:10:09.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:09.444 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:09.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.482+0000 7fc11e9b3080 1 Processor -- start 2023-11-28T11:10:09.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.482+0000 7fc11e9b3080 1 -- start start 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.482+0000 7fc11e9b3080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f7578f0 0x56184f831700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc11e9b3080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f830de0 0x56184f7a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc11e9b3080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 0x56184f8297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc11e9b3080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184f693fe0 con 0x56184f8311b0 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc11e9b3080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184f784790 con 0x56184f830de0 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc11e9b3080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56184f784a70 con 0x56184f7578f0 2023-11-28T11:10:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 0x56184f8297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 0x56184f8297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51112/0 (socket says 172.21.15.83:51112) 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 -- 172.21.15.83:0/2104316869 learned_addr learned my addr 172.21.15.83:0/2104316869 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 -- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f7578f0 msgr2=0x56184f831700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc114ab6700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f7578f0 0x56184f831700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc10ffff700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f830de0 0x56184f7a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f7578f0 0x56184f831700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 -- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f830de0 msgr2=0x56184f7a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f830de0 0x56184f7a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 -- 172.21.15.83:0/2104316869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100005de0 con 0x56184f8311b0 2023-11-28T11:10:09.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc114ab6700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f7578f0 0x56184f831700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:09.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc10ffff700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f830de0 0x56184f7a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:09.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc1152b7700 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 0x56184f8297b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc110002880 tx=0x7fc110002b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afbae07d6cf261d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc10effd700 1 -- 172.21.15.83:0/2104316869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000d6e0 con 0x56184f8311b0 2023-11-28T11:10:09.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc10effd700 1 -- 172.21.15.83:0/2104316869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000d840 con 0x56184f8311b0 2023-11-28T11:10:09.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.483+0000 7fc10effd700 1 -- 172.21.15.83:0/2104316869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000dbf0 con 0x56184f8311b0 2023-11-28T11:10:09.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 -- 172.21.15.83:0/2104316869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 msgr2=0x56184f8297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 0x56184f8297b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc110002880 tx=0x7fc110002b90 comp rx=0 tx=0).stop 2023-11-28T11:10:09.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 -- 172.21.15.83:0/2104316869 shutdown_connections 2023-11-28T11:10:09.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f8311b0 0x56184f8297b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f7578f0 0x56184f831700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/2104316869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f830de0 0x56184f7a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 -- 172.21.15.83:0/2104316869 >> 172.21.15.83:0/2104316869 conn(0x56184f786fb0 msgr2=0x56184f759f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 -- 172.21.15.83:0/2104316869 shutdown_connections 2023-11-28T11:10:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 -- 172.21.15.83:0/2104316869 wait complete. 2023-11-28T11:10:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 Processor -- start 2023-11-28T11:10:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.484+0000 7fc11e9b3080 1 -- start start 2023-11-28T11:10:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc11e9b3080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f7578f0 0x56184f826fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc11e9b3080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f830de0 0x56184f80dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc11e9b3080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f8311b0 0x56184f80e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc11e9b3080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184f79da40 con 0x56184f7578f0 2023-11-28T11:10:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc11e9b3080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184f79dcb0 con 0x56184f8311b0 2023-11-28T11:10:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc11e9b3080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56184f79df90 con 0x56184f830de0 2023-11-28T11:10:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f7578f0 0x56184f826fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f7578f0 0x56184f826fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51114/0 (socket says 172.21.15.83:51114) 2023-11-28T11:10:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 -- 172.21.15.83:0/357473925 learned_addr learned my addr 172.21.15.83:0/357473925 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc10ffff700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f830de0 0x56184f80dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc1152b7700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f8311b0 0x56184f80e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 -- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f830de0 msgr2=0x56184f80dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56184f830de0 0x56184f80dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 -- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f8311b0 msgr2=0x56184f80e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56184f8311b0 0x56184f80e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.485+0000 7fc114ab6700 1 -- 172.21.15.83:0/357473925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110002530 con 0x56184f7578f0 2023-11-28T11:10:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.486+0000 7fc114ab6700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f7578f0 0x56184f826fa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc100015040 tx=0x7fc100002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.486+0000 7fc10dffb700 1 -- 172.21.15.83:0/357473925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100008690 con 0x56184f7578f0 2023-11-28T11:10:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.486+0000 7fc10dffb700 1 -- 172.21.15.83:0/357473925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10001c620 con 0x56184f7578f0 2023-11-28T11:10:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.486+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56184f6ac120 con 0x56184f7578f0 2023-11-28T11:10:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.486+0000 7fc10dffb700 1 -- 172.21.15.83:0/357473925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10001c9d0 con 0x56184f7578f0 2023-11-28T11:10:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.486+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56184f69f830 con 0x56184f7578f0 2023-11-28T11:10:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.487+0000 7fc10dffb700 1 -- 172.21.15.83:0/357473925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc10002c070 con 0x56184f7578f0 2023-11-28T11:10:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.487+0000 7fc10dffb700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0f8066c10 0x7fc0f80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc10dffb700 1 -- 172.21.15.83:0/357473925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc100085e20 con 0x56184f7578f0 2023-11-28T11:10:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc10ffff700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0f8066c10 0x7fc0f80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x56184f8274e0 0x56184f79e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 --> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] -- osd_op(unknown.0.0:1 7.19 7:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x56184f79ecd0 con 0x56184f8274e0 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc10ffff700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0f8066c10 0x7fc0f80690c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56184f784560 tx=0x7fc1080017c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc1152b7700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x56184f8274e0 0x56184f79e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.488+0000 7fc1152b7700 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x56184f8274e0 0x56184f79e790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc1152b7700 1 -- 172.21.15.83:0/357473925 <== osd.6 v2:172.21.15.90:6800/103764 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fc110018070 con 0x56184f8274e0 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x56184f8274e0 msgr2=0x56184f79e790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x56184f8274e0 0x56184f79e790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0f8066c10 msgr2=0x7fc0f80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0f8066c10 0x7fc0f80690c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56184f784560 tx=0x7fc1080017c0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f7578f0 msgr2=0x56184f826fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 --2- 172.21.15.83:0/357473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56184f7578f0 0x56184f826fa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc100015040 tx=0x7fc100002fb0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc1152b7700 1 -- 172.21.15.83:0/357473925 reap_dead start 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 shutdown_connections 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 >> 172.21.15.83:0/357473925 conn(0x56184f786fb0 msgr2=0x56184f833fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 shutdown_connections 2023-11-28T11:10:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.489+0000 7fc11e9b3080 1 -- 172.21.15.83:0/357473925 wait complete. 2023-11-28T11:10:09.506 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2023-11-28T11:10:09.507 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T11:10:09.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:10:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 Processor -- start 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 -- start start 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7d903b0 0x55afd7e53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7e61180 0x55afd7d8d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 0x55afd7d8d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd7cc9fe0 con 0x55afd7e646d0 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd7d8dae0 con 0x55afd7d903b0 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2db2877080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afd7d8dfa0 con 0x55afd7e61180 2023-11-28T11:10:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2da917b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 0x55afd7d8d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.589+0000 7f2da917b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 0x55afd7d8d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51132/0 (socket says 172.21.15.83:51132) 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 -- 172.21.15.83:0/1470356775 learned_addr learned my addr 172.21.15.83:0/1470356775 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da3fff700 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7e61180 0x55afd7d8d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da897a700 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7d903b0 0x55afd7e53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 -- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7e61180 msgr2=0x55afd7d8d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7e61180 0x55afd7d8d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 -- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7d903b0 msgr2=0x55afd7e53050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7d903b0 0x55afd7e53050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 -- 172.21.15.83:0/1470356775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da400b290 con 0x55afd7e646d0 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da917b700 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 0x55afd7d8d5a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2da400b580 tx=0x7f2da400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ec6a6be8cecaea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da2ffd700 1 -- 172.21.15.83:0/1470356775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da401f070 con 0x55afd7e646d0 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da2ffd700 1 -- 172.21.15.83:0/1470356775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da40195d0 con 0x55afd7e646d0 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2da2ffd700 1 -- 172.21.15.83:0/1470356775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4019980 con 0x55afd7e646d0 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 -- 172.21.15.83:0/1470356775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 msgr2=0x55afd7d8d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 0x55afd7d8d5a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2da400b580 tx=0x7f2da400f000 comp rx=0 tx=0).stop 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 -- 172.21.15.83:0/1470356775 shutdown_connections 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e646d0 0x55afd7d8d5a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7d903b0 0x55afd7e53050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 --2- 172.21.15.83:0/1470356775 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7e61180 0x55afd7d8d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 -- 172.21.15.83:0/1470356775 >> 172.21.15.83:0/1470356775 conn(0x55afd7ddaaf0 msgr2=0x55afd7e63110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.590+0000 7f2db2877080 1 -- 172.21.15.83:0/1470356775 shutdown_connections 2023-11-28T11:10:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.591+0000 7f2db2877080 1 -- 172.21.15.83:0/1470356775 wait complete. 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.591+0000 7f2db2877080 1 Processor -- start 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.591+0000 7f2db2877080 1 -- start start 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2db2877080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7d903b0 0x55afd7e5c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2db2877080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e61180 0x55afd7e4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2db2877080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7e646d0 0x55afd7eed020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2db2877080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd7e50310 con 0x55afd7e61180 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2db2877080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd7dbb3b0 con 0x55afd7e646d0 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2db2877080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afd7e5f840 con 0x55afd7d903b0 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e61180 0x55afd7e4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e61180 0x55afd7e4fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51138/0 (socket says 172.21.15.83:51138) 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 -- 172.21.15.83:0/1965113070 learned_addr learned my addr 172.21.15.83:0/1965113070 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da917b700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7e646d0 0x55afd7eed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da897a700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7d903b0 0x55afd7e5c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 -- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7d903b0 msgr2=0x55afd7e5c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7d903b0 0x55afd7e5c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 -- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7e646d0 msgr2=0x55afd7eed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55afd7e646d0 0x55afd7eed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 -- 172.21.15.83:0/1965113070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd7cfc4f0 con 0x55afd7e61180 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da897a700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55afd7d903b0 0x55afd7e5c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da3fff700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e61180 0x55afd7e4fdd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c008730 tx=0x7f2d9c008a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.592+0000 7f2da1ffb700 1 -- 172.21.15.83:0/1965113070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00f040 con 0x55afd7e61180 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.593+0000 7f2da1ffb700 1 -- 172.21.15.83:0/1965113070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c015920 con 0x55afd7e61180 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.593+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55afd7ce2120 con 0x55afd7e61180 2023-11-28T11:10:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.593+0000 7f2da1ffb700 1 -- 172.21.15.83:0/1965113070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c015c90 con 0x55afd7e61180 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.593+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afd7cd5830 con 0x55afd7e61180 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.594+0000 7f2da1ffb700 1 -- 172.21.15.83:0/1965113070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d9c01e400 con 0x55afd7e61180 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.594+0000 7f2da1ffb700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2d8c066c10 0x7f2d8c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.594+0000 7f2da897a700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2d8c066c10 0x7f2d8c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.594+0000 7f2da1ffb700 1 -- 172.21.15.83:0/1965113070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2d9c085780 con 0x55afd7e61180 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.595+0000 7f2db2877080 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x55afd7eef1a0 0x55afd7eef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.595+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 --> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x55afd7e60600 con 0x55afd7eef1a0 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.595+0000 7f2da897a700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2d8c066c10 0x7f2d8c0690c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94001e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.595+0000 7f2da917b700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x55afd7eef1a0 0x55afd7eef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.595+0000 7f2da917b700 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x55afd7eef1a0 0x55afd7eef5a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2da917b700 1 -- 172.21.15.83:0/1965113070 <== osd.6 v2:172.21.15.90:6800/103764 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2da4019b20 con 0x55afd7eef1a0 2023-11-28T11:10:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x55afd7eef1a0 msgr2=0x55afd7eef5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x55afd7eef1a0 0x55afd7eef5a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2d8c066c10 msgr2=0x7f2d8c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2d8c066c10 0x7f2d8c0690c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94001e60 comp rx=0 tx=0).stop 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e61180 msgr2=0x55afd7e4fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 --2- 172.21.15.83:0/1965113070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afd7e61180 0x55afd7e4fdd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c008730 tx=0x7f2d9c008a40 comp rx=0 tx=0).stop 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2da917b700 1 -- 172.21.15.83:0/1965113070 reap_dead start 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 shutdown_connections 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 >> 172.21.15.83:0/1965113070 conn(0x55afd7ddaaf0 msgr2=0x55afd7e630f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 shutdown_connections 2023-11-28T11:10:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.596+0000 7f2db2877080 1 -- 172.21.15.83:0/1965113070 wait complete. 2023-11-28T11:10:09.609 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:10:09.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:09.693 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:09.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:10:09.743 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:10:09.743 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:09.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:09.761 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:10:09.761 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:09.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:09.781 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:10:09.781 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:09.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:09.818 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:10:09.818 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:09.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:09.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:09.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.977+0000 7f4355148700 1 Processor -- start 2023-11-28T11:10:09.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.977+0000 7f4355148700 1 -- start start 2023-11-28T11:10:09.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f4355148700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f435006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f4355148700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435006bd20 con 0x7f435015cd90 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f4355148700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435015d210 con 0x7f435015c9c0 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f4355148700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435006b450 con 0x7f43500b24a0 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434f59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f435015cd90 0x7f43500cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49010/0 (socket says 172.21.15.83:49010) 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434f59e700 1 -- 172.21.15.83:0/2558806998 learned_addr learned my addr 172.21.15.83:0/2558806998 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577006849 0 0) 0x7f435006bd20 con 0x7f435015cd90 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43380035b0 con 0x7f435015cd90 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434e59c700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f435006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2130288256 0 0) 0x7f43380035b0 con 0x7f435015cd90 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 msgr2=0x7f435006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f435006d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 >> v1:172.21.15.90:6789/0 conn(0x7f435015c9c0 legacy=0x7f435006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43501612e0 con 0x7f435015cd90 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f43480019f0 con 0x7f435015cd90 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434e59c700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f435006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 >> v1:172.21.15.83:6789/0 conn(0x7f435015cd90 legacy=0x7f43500cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.978+0000 7f434dd9b700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4338003a30 0x7f4338003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f4338004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338004d90 0x7f4338005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435006b450 con 0x7f4338004d90 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433800bb20 con 0x7f4338003a30 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433800bce0 con 0x7f43500b24a0 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338004d90 0x7f4338005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434ed9d700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4338003a30 0x7f4338003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434e59c700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f4338004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 -- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 msgr2=0x7f4338004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f4338004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 -- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4338003a30 msgr2=0x7f4338003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4338003a30 0x7f4338003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 -- 172.21.15.83:0/2558806998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380035b0 con 0x7f4338004d90 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434e59c700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43500b24a0 0x7f4338004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434f59e700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338004d90 0x7f4338005200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f434800a820 tx=0x7f4338005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37c31a10f4f148c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434800a2f0 con 0x7f4338004d90 2023-11-28T11:10:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348012620 con 0x7f4338004d90 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.979+0000 7f434dd9b700 1 -- 172.21.15.83:0/2558806998 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480129d0 con 0x7f4338004d90 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f4355148700 1 -- 172.21.15.83:0/2558806998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338004d90 msgr2=0x7f4338005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f4355148700 1 --2- 172.21.15.83:0/2558806998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338004d90 0x7f4338005200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f434800a820 tx=0x7f4338005880 comp rx=0 tx=0).stop 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f434f59e700 1 -- 172.21.15.83:0/2558806998 reap_dead start 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f4355148700 1 -- 172.21.15.83:0/2558806998 shutdown_connections 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f4355148700 1 -- 172.21.15.83:0/2558806998 >> 172.21.15.83:0/2558806998 conn(0x7f435006e9f0 msgr2=0x7f43500b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f4355148700 1 -- 172.21.15.83:0/2558806998 shutdown_connections 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.980+0000 7f4355148700 1 -- 172.21.15.83:0/2558806998 wait complete. 2023-11-28T11:10:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.981+0000 7f4355148700 1 Processor -- start 2023-11-28T11:10:09.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.981+0000 7f4355148700 1 -- start start 2023-11-28T11:10:09.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f4355148700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f435015aeb0 0x7f43500c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f4355148700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43500c5690 0x7f435015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f4355148700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 0x7f435015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f4355148700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435015fa10 con 0x7f435015f060 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f4355148700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435015d3a0 con 0x7f43500c5690 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f4355148700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435015b280 con 0x7f435015aeb0 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 0x7f435015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 0x7f435015f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51162/0 (socket says 172.21.15.83:51162) 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 -- 172.21.15.83:0/3577740732 learned_addr learned my addr 172.21.15.83:0/3577740732 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434ed9d700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f435015aeb0 0x7f43500c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434e59c700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43500c5690 0x7f435015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 -- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f435015aeb0 msgr2=0x7f43500c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f435015aeb0 0x7f43500c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 -- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43500c5690 msgr2=0x7f435015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43500c5690 0x7f435015e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 -- 172.21.15.83:0/3577740732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435006b6d0 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.982+0000 7f434f59e700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 0x7f435015f4d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f433800baf0 tx=0x7f43500b2910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.983+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480017b0 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.983+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348001930 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.983+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350161580 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.983+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434801e670 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.983+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435006b9b0 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.983+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4350074320 con 0x7f435015f060 2023-11-28T11:10:09.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.985+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4348010070 con 0x7f435015f060 2023-11-28T11:10:09.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.985+0000 7f43377fe700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f433c0682f0 0x7f433c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.985+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f43480886c0 con 0x7f435015f060 2023-11-28T11:10:09.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.985+0000 7f434ed9d700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f433c0682f0 0x7f433c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.985+0000 7f434ed9d700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f433c0682f0 0x7f433c06a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4344007fd0 tx=0x7f4344009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:09.988+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4348054de0 con 0x7f435015f060 2023-11-28T11:10:10.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.214+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43500b2280 con 0x7f435015f060 2023-11-28T11:10:10.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.215+0000 7f43377fe700 1 -- 172.21.15.83:0/3577740732 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f4348058400 con 0x7f435015f060 2023-11-28T11:10:10.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:10.216 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f433c0682f0 msgr2=0x7f433c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f433c0682f0 0x7f433c06a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4344007fd0 tx=0x7f4344009040 comp rx=0 tx=0).stop 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 msgr2=0x7f435015f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 0x7f435015f4d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f433800baf0 tx=0x7f43500b2910 comp rx=0 tx=0).stop 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 shutdown_connections 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f433c0682f0 0x7f433c06a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435015f060 0x7f435015f4d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f435015aeb0 0x7f43500c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 --2- 172.21.15.83:0/3577740732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43500c5690 0x7f435015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 >> 172.21.15.83:0/3577740732 conn(0x7f435006e9f0 msgr2=0x7f43500b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 shutdown_connections 2023-11-28T11:10:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.217+0000 7f4355148700 1 -- 172.21.15.83:0/3577740732 wait complete. 2023-11-28T11:10:10.220 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:10:10.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:10.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.349+0000 7fc8632b8700 1 Processor -- start 2023-11-28T11:10:10.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.349+0000 7fc8632b8700 1 -- start start 2023-11-28T11:10:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc8632b8700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc85c15c9e0 0x7fc85c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc8632b8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc85c0b5ee0 0x7fc85c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc8632b8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c06b1c0 con 0x7fc85c0b6350 2023-11-28T11:10:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc8632b8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0cda10 con 0x7fc85c0b5ee0 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc8632b8700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0cdcf0 con 0x7fc85c15c9e0 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc860853700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc85c0b5ee0 0x7fc85c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc861855700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc85c0b6350 0x7fc85c0d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49030/0 (socket says 172.21.15.83:49030) 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.350+0000 7fc861855700 1 -- 172.21.15.83:0/1155447303 learned_addr learned my addr 172.21.15.83:0/1155447303 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc861054700 1 --2- 172.21.15.83:0/1155447303 >> v2:172.21.15.90:3300/0 conn(0x7fc85c15c9e0 0x7fc85c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc860853700 1 -- 172.21.15.83:0/1155447303 >> v2:172.21.15.90:3300/0 conn(0x7fc85c15c9e0 msgr2=0x7fc85c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc860853700 1 --2- 172.21.15.83:0/1155447303 >> v2:172.21.15.90:3300/0 conn(0x7fc85c15c9e0 0x7fc85c06ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc860853700 1 -- 172.21.15.83:0/1155447303 >> v1:172.21.15.83:6789/0 conn(0x7fc85c0b6350 legacy=0x7fc85c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc860853700 1 -- 172.21.15.83:0/1155447303 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0d0c60 con 0x7fc85c0b5ee0 2023-11-28T11:10:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc853fff700 1 -- 172.21.15.83:0/1155447303 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759880312 0 0) 0x7fc85c06b1c0 con 0x7fc85c0b6350 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc860853700 1 --2- 172.21.15.83:0/1155447303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0b4b10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc85c066d10 tx=0x7fc84c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c866140429cdf87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc853fff700 1 -- 172.21.15.83:0/1155447303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011830 con 0x7fc85c0b5ee0 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc853fff700 1 -- 172.21.15.83:0/1155447303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c011990 con 0x7fc85c0b5ee0 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.351+0000 7fc853fff700 1 -- 172.21.15.83:0/1155447303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011d20 con 0x7fc85c0b5ee0 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 -- 172.21.15.83:0/1155447303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b5ee0 msgr2=0x7fc85c0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 --2- 172.21.15.83:0/1155447303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0b4b10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc85c066d10 tx=0x7fc84c009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 -- 172.21.15.83:0/1155447303 shutdown_connections 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 --2- 172.21.15.83:0/1155447303 >> v2:172.21.15.90:3300/0 conn(0x7fc85c15c9e0 0x7fc85c06ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 --2- 172.21.15.83:0/1155447303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0b4b10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 -- 172.21.15.83:0/1155447303 >> 172.21.15.83:0/1155447303 conn(0x7fc85c0cfb10 msgr2=0x7fc85c0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 -- 172.21.15.83:0/1155447303 shutdown_connections 2023-11-28T11:10:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.352+0000 7fc8632b8700 1 -- 172.21.15.83:0/1155447303 wait complete. 2023-11-28T11:10:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.353+0000 7fc8632b8700 1 Processor -- start 2023-11-28T11:10:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.353+0000 7fc8632b8700 1 -- start start 2023-11-28T11:10:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc8632b8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc8632b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 0x7fc85c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc8632b8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc85c15c9e0 0x7fc85c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc8632b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0ccb60 con 0x7fc85c0b6350 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc8632b8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c158600 con 0x7fc85c0b5ee0 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc8632b8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c15a2a0 con 0x7fc85c15c9e0 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 0x7fc85c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 0x7fc85c1580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51174/0 (socket says 172.21.15.83:51174) 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 -- 172.21.15.83:0/850148443 learned_addr learned my addr 172.21.15.83:0/850148443 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc861855700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc85c15c9e0 0x7fc85c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc861054700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 -- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc85c15c9e0 msgr2=0x7fc85c159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc85c15c9e0 0x7fc85c159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 -- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc85c0b5ee0 msgr2=0x7fc85c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 -- 172.21.15.83:0/850148443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c0094a0 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc861855700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc85c15c9e0 0x7fc85c159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.354+0000 7fc860853700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 0x7fc85c1580c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc85c06d200 tx=0x7fc84c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.355+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0036a0 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.355+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c01f070 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.355+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c0d1220 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.355+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c003a20 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.355+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c0d0f40 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.356+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c0cd6e0 con 0x7fc85c0b6350 2023-11-28T11:10:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.356+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc84c00f050 con 0x7fc85c0b6350 2023-11-28T11:10:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.356+0000 7fc851ffb700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc8400682f0 0x7fc84006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.356+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc84c021070 con 0x7fc85c0b6350 2023-11-28T11:10:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.357+0000 7fc861054700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc8400682f0 0x7fc84006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.357+0000 7fc861054700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc8400682f0 0x7fc84006a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc858009b40 tx=0x7fc858009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.361+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc84c053dc0 con 0x7fc85c0b6350 2023-11-28T11:10:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.545+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc85c074320 con 0x7fc85c0b6350 2023-11-28T11:10:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.546+0000 7fc851ffb700 1 -- 172.21.15.83:0/850148443 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fc84c0573e0 con 0x7fc85c0b6350 2023-11-28T11:10:10.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:10.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:47.730175+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:10.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc8400682f0 msgr2=0x7fc84006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc8400682f0 0x7fc84006a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc858009b40 tx=0x7fc858009210 comp rx=0 tx=0).stop 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 msgr2=0x7fc85c1580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 0x7fc85c1580c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc85c06d200 tx=0x7fc84c004750 comp rx=0 tx=0).stop 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 shutdown_connections 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc85c15c9e0 0x7fc85c159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc85c0b5ee0 0x7fc85c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc8400682f0 0x7fc84006a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 --2- 172.21.15.83:0/850148443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85c0b6350 0x7fc85c1580c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 >> 172.21.15.83:0/850148443 conn(0x7fc85c0cfb10 msgr2=0x7fc85c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 shutdown_connections 2023-11-28T11:10:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.548+0000 7fc8632b8700 1 -- 172.21.15.83:0/850148443 wait complete. 2023-11-28T11:10:10.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:09:44.997179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:09:45.687674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:10.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.683+0000 7fb1a8f81700 1 Processor -- start 2023-11-28T11:10:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.683+0000 7fb1a8f81700 1 -- start start 2023-11-28T11:10:10.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a8f81700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb1a415c890 0x7fb1a40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a8f81700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a40b4bb0 0x7fb1a406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a8f81700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a406d4a0 con 0x7fb1a406e1a0 2023-11-28T11:10:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a8f81700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a406e620 con 0x7fb1a415c890 2023-11-28T11:10:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a8f81700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40b5020 con 0x7fb1a40b4bb0 2023-11-28T11:10:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a259c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb1a415c890 0x7fb1a40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a1d9b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a40b4bb0 0x7fb1a406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a2d9d700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fb1a406e1a0 0x7fb1a41604e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:47390/0 (socket says 172.21.15.83:47390) 2023-11-28T11:10:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a2d9d700 1 -- 172.21.15.83:0/267883561 learned_addr learned my addr 172.21.15.83:0/267883561 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a259c700 1 -- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a40b4bb0 msgr2=0x7fb1a406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a259c700 1 --2- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a40b4bb0 0x7fb1a406dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a259c700 1 -- 172.21.15.83:0/267883561 >> v1:172.21.15.90:6789/0 conn(0x7fb1a406e1a0 legacy=0x7fb1a41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a159a700 1 -- 172.21.15.83:0/267883561 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566837598 0 0) 0x7fb1a406d4a0 con 0x7fb1a406e1a0 2023-11-28T11:10:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a259c700 1 -- 172.21.15.83:0/267883561 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a406c160 con 0x7fb1a415c890 2023-11-28T11:10:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.684+0000 7fb1a1d9b700 1 --2- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a40b4bb0 0x7fb1a406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a259c700 1 --2- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a415c890 0x7fb1a40aee60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb198009850 tx=0x7fb198009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b4c3c61e6f3d0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a159a700 1 -- 172.21.15.83:0/267883561 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198005e30 con 0x7fb1a415c890 2023-11-28T11:10:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a159a700 1 -- 172.21.15.83:0/267883561 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198006030 con 0x7fb1a415c890 2023-11-28T11:10:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a159a700 1 -- 172.21.15.83:0/267883561 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980063c0 con 0x7fb1a415c890 2023-11-28T11:10:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 -- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a415c890 msgr2=0x7fb1a40aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a415c890 0x7fb1a40aee60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb198009850 tx=0x7fb198009b60 comp rx=0 tx=0).stop 2023-11-28T11:10:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 -- 172.21.15.83:0/267883561 shutdown_connections 2023-11-28T11:10:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a415c890 0x7fb1a40aee60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/267883561 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a40b4bb0 0x7fb1a406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 -- 172.21.15.83:0/267883561 >> 172.21.15.83:0/267883561 conn(0x7fb1a40b2b60 msgr2=0x7fb1a40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 -- 172.21.15.83:0/267883561 shutdown_connections 2023-11-28T11:10:10.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.685+0000 7fb1a8f81700 1 -- 172.21.15.83:0/267883561 wait complete. 2023-11-28T11:10:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.687+0000 7fb1a8f81700 1 Processor -- start 2023-11-28T11:10:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.687+0000 7fb1a8f81700 1 -- start start 2023-11-28T11:10:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a8f81700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a406e1a0 0x7fb1a40cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a8f81700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a40b4bb0 0x7fb1a40c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a8f81700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 0x7fb1a40c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a8f81700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c4180 con 0x7fb1a415c890 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a8f81700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c4ae0 con 0x7fb1a40b4bb0 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a8f81700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c6210 con 0x7fb1a406e1a0 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 0x7fb1a40c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 0x7fb1a40c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51190/0 (socket says 172.21.15.83:51190) 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a2d9d700 1 -- 172.21.15.83:0/449547958 learned_addr learned my addr 172.21.15.83:0/449547958 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a259c700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a406e1a0 0x7fb1a40cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.688+0000 7fb1a1d9b700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a40b4bb0 0x7fb1a40c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a2d9d700 1 -- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a406e1a0 msgr2=0x7fb1a40cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a2d9d700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a406e1a0 0x7fb1a40cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a2d9d700 1 -- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a40b4bb0 msgr2=0x7fb1a40c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a2d9d700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a40b4bb0 0x7fb1a40c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a2d9d700 1 -- 172.21.15.83:0/449547958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940094a0 con 0x7fb1a415c890 2023-11-28T11:10:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a2d9d700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 0x7fb1a40c5cd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00cb90 tx=0x7fb19c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00ab30 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00ac90 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb198009430 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0070d0 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.689+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a406c400 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.690+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a40b5ae0 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.690+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb19c01a030 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.691+0000 7fb192ffd700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb1840682f0 0x7fb18406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.691+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb19c085a40 con 0x7fb1a415c890 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.691+0000 7fb1a259c700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb1840682f0 0x7fb18406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.691+0000 7fb1a259c700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb1840682f0 0x7fb18406a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb198013040 tx=0x7fb198008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.695+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb19c052160 con 0x7fb1a415c890 2023-11-28T11:10:10.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.883+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb1a40d0bb0 con 0x7fb1a415c890 2023-11-28T11:10:10.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.883+0000 7fb192ffd700 1 -- 172.21.15.83:0/449547958 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb19c0180a0 con 0x7fb1a415c890 2023-11-28T11:10:10.885 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:10:10.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb1840682f0 msgr2=0x7fb18406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb1840682f0 0x7fb18406a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb198013040 tx=0x7fb198008070 comp rx=0 tx=0).stop 2023-11-28T11:10:10.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 msgr2=0x7fb1a40c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 0x7fb1a40c5cd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00cb90 tx=0x7fb19c00cf50 comp rx=0 tx=0).stop 2023-11-28T11:10:10.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 shutdown_connections 2023-11-28T11:10:10.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb1840682f0 0x7fb18406a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1a415c890 0x7fb1a40c5cd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1a406e1a0 0x7fb1a40cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 --2- 172.21.15.83:0/449547958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a40b4bb0 0x7fb1a40c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 >> 172.21.15.83:0/449547958 conn(0x7fb1a40b2b60 msgr2=0x7fb1a40cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 shutdown_connections 2023-11-28T11:10:10.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:10.885+0000 7fb1a8f81700 1 -- 172.21.15.83:0/449547958 wait complete. 2023-11-28T11:10:10.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.020+0000 7f808695f700 1 Processor -- start 2023-11-28T11:10:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.021+0000 7f808695f700 1 -- start start 2023-11-28T11:10:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.021+0000 7f808695f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f808015c890 0x7f80800aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.021+0000 7f808695f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80800b4bb0 0x7f808006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.021+0000 7f808695f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808006d4a0 con 0x7f808006e1a0 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.021+0000 7f808695f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808006e620 con 0x7f808015c890 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.021+0000 7f808695f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800b5020 con 0x7f80800b4bb0 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80800b4bb0 0x7f808006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807ffff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f808015c890 0x7f80800aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f8084efc700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f808006e1a0 0x7f80801604e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:47404/0 (socket says 172.21.15.83:47404) 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f8084efc700 1 -- 172.21.15.83:0/2443118765 learned_addr learned my addr 172.21.15.83:0/2443118765 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807f7fe700 1 -- 172.21.15.83:0/2443118765 >> v2:172.21.15.90:3300/0 conn(0x7f808015c890 msgr2=0x7f80800aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807f7fe700 1 --2- 172.21.15.83:0/2443118765 >> v2:172.21.15.90:3300/0 conn(0x7f808015c890 0x7f80800aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807f7fe700 1 -- 172.21.15.83:0/2443118765 >> v1:172.21.15.90:6789/0 conn(0x7f808006e1a0 legacy=0x7f80801604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807f7fe700 1 -- 172.21.15.83:0/2443118765 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808006c160 con 0x7f80800b4bb0 2023-11-28T11:10:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807ffff700 1 --2- 172.21.15.83:0/2443118765 >> v2:172.21.15.90:3300/0 conn(0x7f808015c890 0x7f80800aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807f7fe700 1 --2- 172.21.15.83:0/2443118765 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80800b4bb0 0x7f808006dc60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8070009870 tx=0x7f8070009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242408cd62ec1aa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807effd700 1 -- 172.21.15.83:0/2443118765 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070011830 con 0x7f80800b4bb0 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807effd700 1 -- 172.21.15.83:0/2443118765 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070011990 con 0x7f80800b4bb0 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.022+0000 7f807effd700 1 -- 172.21.15.83:0/2443118765 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070011d20 con 0x7f80800b4bb0 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 -- 172.21.15.83:0/2443118765 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80800b4bb0 msgr2=0x7f808006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 --2- 172.21.15.83:0/2443118765 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80800b4bb0 0x7f808006dc60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8070009870 tx=0x7f8070009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 -- 172.21.15.83:0/2443118765 shutdown_connections 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 --2- 172.21.15.83:0/2443118765 >> v2:172.21.15.90:3300/0 conn(0x7f808015c890 0x7f80800aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 --2- 172.21.15.83:0/2443118765 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f80800b4bb0 0x7f808006dc60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 -- 172.21.15.83:0/2443118765 >> 172.21.15.83:0/2443118765 conn(0x7f80800b2b60 msgr2=0x7f80800b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 -- 172.21.15.83:0/2443118765 shutdown_connections 2023-11-28T11:10:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.023+0000 7f808695f700 1 -- 172.21.15.83:0/2443118765 wait complete. 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.024+0000 7f808695f700 1 Processor -- start 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.024+0000 7f808695f700 1 -- start start 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f808695f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 0x7f80800c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f808695f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80800b4bb0 0x7f8080158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f808695f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f808015c890 0x7f80800c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f808695f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800c5150 con 0x7f808006e1a0 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f808695f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800c8980 con 0x7f80800b4bb0 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f808695f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800c4e70 con 0x7f808015c890 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 0x7f80800c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 0x7f80800c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51200/0 (socket says 172.21.15.83:51200) 2023-11-28T11:10:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 -- 172.21.15.83:0/546484845 learned_addr learned my addr 172.21.15.83:0/546484845 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f8084efc700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f808015c890 0x7f80800c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807f7fe700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80800b4bb0 0x7f8080158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 -- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f808015c890 msgr2=0x7f80800c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f808015c890 0x7f80800c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 -- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80800b4bb0 msgr2=0x7f8080158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80800b4bb0 0x7f8080158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.025+0000 7f807ffff700 1 -- 172.21.15.83:0/546484845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074009430 con 0x7f808006e1a0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.026+0000 7f807ffff700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 0x7f80800c9f20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f807400ebe0 tx=0x7f8074009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.026+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074005a20 con 0x7f808006e1a0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.026+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074005b80 con 0x7f808006e1a0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.026+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80700094a0 con 0x7f808006e1a0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.026+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400c8b0 con 0x7f808006e1a0 2023-11-28T11:10:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.026+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808006c400 con 0x7f808006e1a0 2023-11-28T11:10:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.027+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80800b5ae0 con 0x7f808006e1a0 2023-11-28T11:10:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.027+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8074022070 con 0x7f808006e1a0 2023-11-28T11:10:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.027+0000 7f807cff9700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f806c0682f0 0x7f806c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.027+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8074085f80 con 0x7f808006e1a0 2023-11-28T11:10:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.028+0000 7f807f7fe700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f806c0682f0 0x7f806c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.028+0000 7f807f7fe700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f806c0682f0 0x7f806c06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f80700090b0 tx=0x7f807001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.032+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80740526a0 con 0x7f808006e1a0 2023-11-28T11:10:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.213+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80800c8c60 con 0x7f808006e1a0 2023-11-28T11:10:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.214+0000 7f807cff9700 1 -- 172.21.15.83:0/546484845 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8074020020 con 0x7f808006e1a0 2023-11-28T11:10:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:10:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.215+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f806c0682f0 msgr2=0x7f806c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.215+0000 7f808695f700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f806c0682f0 0x7f806c06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f80700090b0 tx=0x7f807001c040 comp rx=0 tx=0).stop 2023-11-28T11:10:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.215+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 msgr2=0x7f80800c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.215+0000 7f808695f700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 0x7f80800c9f20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f807400ebe0 tx=0x7f8074009e00 comp rx=0 tx=0).stop 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 shutdown_connections 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f808015c890 0x7f80800c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f806c0682f0 0x7f806c06a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808006e1a0 0x7f80800c9f20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 --2- 172.21.15.83:0/546484845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80800b4bb0 0x7f8080158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 >> 172.21.15.83:0/546484845 conn(0x7f80800b2b60 msgr2=0x7f80800b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 shutdown_connections 2023-11-28T11:10:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.216+0000 7f808695f700 1 -- 172.21.15.83:0/546484845 wait complete. 2023-11-28T11:10:11.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:10:11.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.351+0000 7f97accd5700 1 Processor -- start 2023-11-28T11:10:11.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.351+0000 7f97accd5700 1 -- start start 2023-11-28T11:10:11.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97accd5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 0x7f97a8159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97accd5700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f97a8158cd0 0x7f97a80cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97accd5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a80c9e70 con 0x7f97a80cb650 2023-11-28T11:10:11.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97accd5700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a815a000 con 0x7f97a8158cd0 2023-11-28T11:10:11.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97accd5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a815c790 con 0x7f97a8158900 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97a6d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f97a80cb650 0x7f97a806d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49070/0 (socket says 172.21.15.83:49070) 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97a6d9d700 1 -- 172.21.15.83:0/1436928401 learned_addr learned my addr 172.21.15.83:0/1436928401 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97a5d9b700 1 --2- 172.21.15.83:0/1436928401 >> v2:172.21.15.90:3300/0 conn(0x7f97a8158cd0 0x7f97a80cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97a659c700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 0x7f97a8159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.352+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981253397 0 0) 0x7f97a80c9e70 con 0x7f97a80cb650 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97900035b0 con 0x7f97a80cb650 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 548939601 0 0) 0x7f97900035b0 con 0x7f97a80cb650 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 msgr2=0x7f97a8159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 0x7f97a8159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 >> v2:172.21.15.90:3300/0 conn(0x7f97a8158cd0 msgr2=0x7f97a80cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 --2- 172.21.15.83:0/1436928401 >> v2:172.21.15.90:3300/0 conn(0x7f97a8158cd0 0x7f97a80cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a815ca70 con 0x7f97a80cb650 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f97a00019f0 con 0x7f97a80cb650 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 >> v1:172.21.15.83:6789/0 conn(0x7f97a80cb650 legacy=0x7f97a806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9790003a00 0x7f9790003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9790004850 0x7f9790004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 0x7f9790005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a815a000 con 0x7f9790003a00 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a815c790 con 0x7f9790004850 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979000ba90 con 0x7f97a8158900 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9790003a00 0x7f9790003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a5d9b700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9790004850 0x7f9790004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a6d9d700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 0x7f9790005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 -- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 msgr2=0x7f9790005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97a8158900 0x7f9790005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 -- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9790004850 msgr2=0x7f9790004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9790004850 0x7f9790004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 -- 172.21.15.83:0/1436928401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900035b0 con 0x7f9790003a00 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a659c700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9790003a00 0x7f9790003e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f979c009b60 tx=0x7f9790005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd3f7250abfbec5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00e6c0 con 0x7f9790003a00 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.353+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00e820 con 0x7f9790003a00 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97a559a700 1 -- 172.21.15.83:0/1436928401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00eb90 con 0x7f9790003a00 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97accd5700 1 -- 172.21.15.83:0/1436928401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9790003a00 msgr2=0x7f9790003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97accd5700 1 --2- 172.21.15.83:0/1436928401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9790003a00 0x7f9790003e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f979c009b60 tx=0x7f9790005880 comp rx=0 tx=0).stop 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97a6d9d700 1 -- 172.21.15.83:0/1436928401 reap_dead start 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97accd5700 1 -- 172.21.15.83:0/1436928401 shutdown_connections 2023-11-28T11:10:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97accd5700 1 -- 172.21.15.83:0/1436928401 >> 172.21.15.83:0/1436928401 conn(0x7f97a80b1090 msgr2=0x7f97a80b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97accd5700 1 -- 172.21.15.83:0/1436928401 shutdown_connections 2023-11-28T11:10:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.354+0000 7f97accd5700 1 -- 172.21.15.83:0/1436928401 wait complete. 2023-11-28T11:10:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.356+0000 7f97accd5700 1 Processor -- start 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.356+0000 7f97accd5700 1 -- start start 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97accd5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9790003a00 0x7f97a80b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97accd5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 0x7f97a81f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97accd5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97a80c6430 0x7f97a81f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97accd5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b2250 con 0x7f97a81515e0 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97accd5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a8151a70 con 0x7f97a80c6430 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97accd5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b24f0 con 0x7f9790003a00 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 0x7f97a81f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 0x7f97a81f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51220/0 (socket says 172.21.15.83:51220) 2023-11-28T11:10:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 -- 172.21.15.83:0/3915781630 learned_addr learned my addr 172.21.15.83:0/3915781630 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a659c700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9790003a00 0x7f97a80b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a6d9d700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97a80c6430 0x7f97a81f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 -- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9790003a00 msgr2=0x7f97a80b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9790003a00 0x7f97a80b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 -- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97a80c6430 msgr2=0x7f97a81f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97a80c6430 0x7f97a81f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 -- 172.21.15.83:0/3915781630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a815d030 con 0x7f97a81515e0 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a659c700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9790003a00 0x7f97a80b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.357+0000 7f97a5d9b700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 0x7f97a81f3aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9798011aa0 tx=0x7f9798011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.358+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979800f9e0 con 0x7f97a81515e0 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.358+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979800fb40 con 0x7f97a81515e0 2023-11-28T11:10:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.358+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a815cd50 con 0x7f97a81515e0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.358+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798014440 con 0x7f97a81515e0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.358+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a815d310 con 0x7f97a81515e0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.359+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a80742b0 con 0x7f97a81515e0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.359+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97980145a0 con 0x7f97a81515e0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.359+0000 7f9796ffd700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97880682f0 0x7f978806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.359+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9798086620 con 0x7f97a81515e0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.360+0000 7f97a659c700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97880682f0 0x7f978806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.360+0000 7f97a659c700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97880682f0 0x7f978806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f979c0090b0 tx=0x7f979c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.364+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9798052ce0 con 0x7f97a81515e0 2023-11-28T11:10:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.545+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97a8072460 con 0x7f97a81515e0 2023-11-28T11:10:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.546+0000 7f9796ffd700 1 -- 172.21.15.83:0/3915781630 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9798056300 con 0x7f97a81515e0 2023-11-28T11:10:11.547 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:10:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97880682f0 msgr2=0x7f978806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97880682f0 0x7f978806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f979c0090b0 tx=0x7f979c008030 comp rx=0 tx=0).stop 2023-11-28T11:10:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 msgr2=0x7f97a81f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 0x7f97a81f3aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9798011aa0 tx=0x7f9798011e60 comp rx=0 tx=0).stop 2023-11-28T11:10:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 shutdown_connections 2023-11-28T11:10:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97a80c6430 0x7f97a81f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9790003a00 0x7f97a80b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97880682f0 0x7f978806a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 --2- 172.21.15.83:0/3915781630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a81515e0 0x7f97a81f3aa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.547+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 >> 172.21.15.83:0/3915781630 conn(0x7f97a80b1090 msgr2=0x7f97a806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.548+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 shutdown_connections 2023-11-28T11:10:11.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.548+0000 7f97accd5700 1 -- 172.21.15.83:0/3915781630 wait complete. 2023-11-28T11:10:11.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:10:11.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.681+0000 7f951a622700 1 Processor -- start 2023-11-28T11:10:11.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f951a622700 1 -- start start 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f951a622700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f951406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f951a622700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f951415ac30 0x7f951415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f951a622700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951406be70 con 0x7f95140cbe30 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f951a622700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951415ec90 con 0x7f951415ac30 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f951a622700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95140cb270 con 0x7f951406b4e0 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f9518bbf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f951415ac30 0x7f951415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f9518bbf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f951415ac30 0x7f951415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51230/0 (socket says 172.21.15.83:51230) 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.682+0000 7f9518bbf700 1 -- 172.21.15.83:0/1334104441 learned_addr learned my addr 172.21.15.83:0/1334104441 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f95137fe700 1 --2- 172.21.15.83:0/1334104441 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f951406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9518bbf700 1 -- 172.21.15.83:0/1334104441 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 msgr2=0x7f951406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9518bbf700 1 --2- 172.21.15.83:0/1334104441 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f951406b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9518bbf700 1 -- 172.21.15.83:0/1334104441 >> v1:172.21.15.90:6789/0 conn(0x7f95140cbe30 legacy=0x7f95140b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9518bbf700 1 -- 172.21.15.83:0/1334104441 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951415e380 con 0x7f951415ac30 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9518bbf700 1 --2- 172.21.15.83:0/1334104441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f951415b0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f950c009e50 tx=0x7f950c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2944626bed81b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9512ffd700 1 -- 172.21.15.83:0/1334104441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00e070 con 0x7f951415ac30 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9512ffd700 1 -- 172.21.15.83:0/1334104441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c003a20 con 0x7f951415ac30 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.683+0000 7f9512ffd700 1 -- 172.21.15.83:0/1334104441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c003dd0 con 0x7f951415ac30 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 -- 172.21.15.83:0/1334104441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 msgr2=0x7f951415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 --2- 172.21.15.83:0/1334104441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f951415b0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f950c009e50 tx=0x7f950c00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 -- 172.21.15.83:0/1334104441 shutdown_connections 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 --2- 172.21.15.83:0/1334104441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f951415b0b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 --2- 172.21.15.83:0/1334104441 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f951406b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 -- 172.21.15.83:0/1334104441 >> 172.21.15.83:0/1334104441 conn(0x7f95140cc690 msgr2=0x7f95140cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 -- 172.21.15.83:0/1334104441 shutdown_connections 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.684+0000 7f951a622700 1 -- 172.21.15.83:0/1334104441 wait complete. 2023-11-28T11:10:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.685+0000 7f951a622700 1 Processor -- start 2023-11-28T11:10:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.685+0000 7f951a622700 1 -- start start 2023-11-28T11:10:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f95140c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95140cbe30 0x7f95141e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f9514151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140c6ce0 con 0x7f951415ac30 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140c72a0 con 0x7f95140cbe30 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95141e5650 con 0x7f951406b4e0 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f9514151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f9514151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51236/0 (socket says 172.21.15.83:51236) 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 -- 172.21.15.83:0/393379922 learned_addr learned my addr 172.21.15.83:0/393379922 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f95137fe700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95140cbe30 0x7f95141e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9513fff700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f95140c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 -- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 msgr2=0x7f95140c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f95140c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 -- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95140cbe30 msgr2=0x7f95141e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95140cbe30 0x7f95141e4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 -- 172.21.15.83:0/393379922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c009b30 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f95137fe700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95140cbe30 0x7f95141e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9518bbf700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f9514151980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b730 tx=0x7f950c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c004240 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c0043a0 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951415e640 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.686+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c004750 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.687+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95140b2110 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.687+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951415e920 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.688+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f950c02c070 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.688+0000 7f9510ff9700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94f40682f0 0x7f94f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.688+0000 7f9513fff700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94f40682f0 0x7f94f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.688+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f950c0870e0 con 0x7f951415ac30 2023-11-28T11:10:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.689+0000 7f9513fff700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94f40682f0 0x7f94f406a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f94fc004f30 tx=0x7f94fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.693+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f950c053710 con 0x7f951415ac30 2023-11-28T11:10:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.874+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f951406be70 con 0x7f951415ac30 2023-11-28T11:10:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.875+0000 7f9510ff9700 1 -- 172.21.15.83:0/393379922 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f950c020020 con 0x7f951415ac30 2023-11-28T11:10:11.876 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:10:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94f40682f0 msgr2=0x7f94f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94f40682f0 0x7f94f406a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f94fc004f30 tx=0x7f94fc009040 comp rx=0 tx=0).stop 2023-11-28T11:10:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 msgr2=0x7f9514151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f9514151980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b730 tx=0x7f950c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 shutdown_connections 2023-11-28T11:10:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f94f40682f0 0x7f94f406a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f951415ac30 0x7f9514151980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f951406b4e0 0x7f95140c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 --2- 172.21.15.83:0/393379922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95140cbe30 0x7f95141e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 >> 172.21.15.83:0/393379922 conn(0x7f95140cc690 msgr2=0x7f95140ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 shutdown_connections 2023-11-28T11:10:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:11.877+0000 7f951a622700 1 -- 172.21.15.83:0/393379922 wait complete. 2023-11-28T11:10:11.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:12.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.009+0000 7fad053d2700 1 Processor -- start 2023-11-28T11:10:12.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.009+0000 7fad053d2700 1 -- start start 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7fad053d2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad0015a4f0 0x7fad00071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7fad053d2700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fad000cd540 0x7fad000cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7fad053d2700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000cdeb0 con 0x7fad00160d40 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7fad053d2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad001612c0 con 0x7fad0015a4f0 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7fad053d2700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad000d0a60 con 0x7fad000cd540 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad0015a4f0 0x7fad00071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad0015a4f0 0x7fad00071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51240/0 (socket says 172.21.15.83:51240) 2023-11-28T11:10:12.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 -- 172.21.15.83:0/1846077057 learned_addr learned my addr 172.21.15.83:0/1846077057 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:12.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facff7fe700 1 --2- 172.21.15.83:0/1846077057 >> v2:172.21.15.90:3300/0 conn(0x7fad000cd540 0x7fad000cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 -- 172.21.15.83:0/1846077057 >> v2:172.21.15.90:3300/0 conn(0x7fad000cd540 msgr2=0x7fad000cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 --2- 172.21.15.83:0/1846077057 >> v2:172.21.15.90:3300/0 conn(0x7fad000cd540 0x7fad000cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 -- 172.21.15.83:0/1846077057 >> v1:172.21.15.90:6789/0 conn(0x7fad00160d40 legacy=0x7fad000cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 -- 172.21.15.83:0/1846077057 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0015ce50 con 0x7fad0015a4f0 2023-11-28T11:10:12.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.010+0000 7facfeffd700 1 --2- 172.21.15.83:0/1846077057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0015a4f0 0x7fad00071380 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fad000cd2d0 tx=0x7facf4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfb6aaf9e45f9c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7facfdffb700 1 -- 172.21.15.83:0/1846077057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4004230 con 0x7fad0015a4f0 2023-11-28T11:10:12.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7facfdffb700 1 -- 172.21.15.83:0/1846077057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4004390 con 0x7fad0015a4f0 2023-11-28T11:10:12.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7facfdffb700 1 -- 172.21.15.83:0/1846077057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40036a0 con 0x7fad0015a4f0 2023-11-28T11:10:12.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 -- 172.21.15.83:0/1846077057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0015a4f0 msgr2=0x7fad00071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 --2- 172.21.15.83:0/1846077057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0015a4f0 0x7fad00071380 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fad000cd2d0 tx=0x7facf4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 -- 172.21.15.83:0/1846077057 shutdown_connections 2023-11-28T11:10:12.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 --2- 172.21.15.83:0/1846077057 >> v2:172.21.15.90:3300/0 conn(0x7fad000cd540 0x7fad000cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 --2- 172.21.15.83:0/1846077057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0015a4f0 0x7fad00071380 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 -- 172.21.15.83:0/1846077057 >> 172.21.15.83:0/1846077057 conn(0x7fad0006f300 msgr2=0x7fad0006f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 -- 172.21.15.83:0/1846077057 shutdown_connections 2023-11-28T11:10:12.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.011+0000 7fad053d2700 1 -- 172.21.15.83:0/1846077057 wait complete. 2023-11-28T11:10:12.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.012+0000 7fad053d2700 1 Processor -- start 2023-11-28T11:10:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.012+0000 7fad053d2700 1 -- start start 2023-11-28T11:10:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7fad053d2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 0x7fad00157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7fad053d2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fad0015a4f0 0x7fad0006bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7fad053d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad00160d40 0x7fad0006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7fad053d2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000b3d10 con 0x7fad000cd540 2023-11-28T11:10:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7fad053d2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000b3fb0 con 0x7fad00160d40 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7fad053d2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000b4290 con 0x7fad0015a4f0 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 0x7fad00157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 0x7fad00157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51246/0 (socket says 172.21.15.83:51246) 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 -- 172.21.15.83:0/2447455281 learned_addr learned my addr 172.21.15.83:0/2447455281 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfe7fc700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fad0015a4f0 0x7fad0006bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facff7fe700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad00160d40 0x7fad0006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 -- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fad0015a4f0 msgr2=0x7fad0006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fad0015a4f0 0x7fad0006bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 -- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad00160d40 msgr2=0x7fad0006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad00160d40 0x7fad0006c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.013+0000 7facfeffd700 1 -- 172.21.15.83:0/2447455281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4009550 con 0x7fad000cd540 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7facfe7fc700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fad0015a4f0 0x7fad0006bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7facff7fe700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad00160d40 0x7fad0006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7facfeffd700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 0x7fad00157440 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fad000ce040 tx=0x7facf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4003d80 con 0x7fad000cd540 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf401f070 con 0x7fad000cd540 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4011870 con 0x7fad000cd540 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad000cee00 con 0x7fad000cd540 2023-11-28T11:10:12.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.014+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad000ceb20 con 0x7fad000cd540 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.015+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad000cbb10 con 0x7fad000cd540 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.015+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7facf4017020 con 0x7fad000cd540 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.015+0000 7face77fe700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7facec0682f0 0x7facec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.016+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7facf40872c0 con 0x7fad000cd540 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.016+0000 7facfe7fc700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7facec0682f0 0x7facec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.016+0000 7facfe7fc700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7facec0682f0 0x7facec06a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad000c7bd0 tx=0x7facf0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.020+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7facf40539e0 con 0x7fad000cd540 2023-11-28T11:10:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.243+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad00074320 con 0x7fad000cd540 2023-11-28T11:10:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.244+0000 7face77fe700 1 -- 172.21.15.83:0/2447455281 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7facf4057000 con 0x7fad000cd540 2023-11-28T11:10:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:12.245 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:12.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7facec0682f0 msgr2=0x7facec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7facec0682f0 0x7facec06a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad000c7bd0 tx=0x7facf0009040 comp rx=0 tx=0).stop 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 msgr2=0x7fad00157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 0x7fad00157440 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fad000ce040 tx=0x7facf4004750 comp rx=0 tx=0).stop 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 shutdown_connections 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad00160d40 0x7fad0006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7facec0682f0 0x7facec06a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad000cd540 0x7fad00157440 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 --2- 172.21.15.83:0/2447455281 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fad0015a4f0 0x7fad0006bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 >> 172.21.15.83:0/2447455281 conn(0x7fad0006f300 msgr2=0x7fad0015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 shutdown_connections 2023-11-28T11:10:12.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.246+0000 7fad053d2700 1 -- 172.21.15.83:0/2447455281 wait complete. 2023-11-28T11:10:12.249 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:10:12.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:12.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 Processor -- start 2023-11-28T11:10:12.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 -- start start 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f13d8071710 0x7f13d80cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13d81586b0 0x7f13d8158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80cac90 con 0x7f13d8071ae0 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80ca3f0 con 0x7f13d81586b0 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13dd597700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80ca6d0 con 0x7f13d8071710 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.378+0000 7f13d77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13d81586b0 0x7f13d8158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13d81586b0 0x7f13d8158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51256/0 (socket says 172.21.15.83:51256) 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 -- 172.21.15.83:0/712496358 learned_addr learned my addr 172.21.15.83:0/712496358 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d67fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f13d8071ae0 0x7f13d80c98a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49114/0 (socket says 172.21.15.83:49114) 2023-11-28T11:10:12.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/712496358 >> v2:172.21.15.90:3300/0 conn(0x7f13d8071710 0x7f13d80cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 -- 172.21.15.83:0/712496358 >> v2:172.21.15.90:3300/0 conn(0x7f13d8071710 msgr2=0x7f13d80cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 --2- 172.21.15.83:0/712496358 >> v2:172.21.15.90:3300/0 conn(0x7f13d8071710 0x7f13d80cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 -- 172.21.15.83:0/712496358 >> v1:172.21.15.83:6789/0 conn(0x7f13d8071ae0 legacy=0x7f13d80c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d5ffb700 1 -- 172.21.15.83:0/712496358 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812861534 0 0) 0x7f13d80cac90 con 0x7f13d8071ae0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 -- 172.21.15.83:0/712496358 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80ca9b0 con 0x7f13d81586b0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d77fe700 1 --2- 172.21.15.83:0/712496358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d81586b0 0x7f13d8158aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f13d8072030 tx=0x7f13d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1238cb35d467b3f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d5ffb700 1 -- 172.21.15.83:0/712496358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000e070 con 0x7f13d81586b0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d5ffb700 1 -- 172.21.15.83:0/712496358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d000b9c0 con 0x7f13d81586b0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.379+0000 7f13d5ffb700 1 -- 172.21.15.83:0/712496358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000bd70 con 0x7f13d81586b0 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 -- 172.21.15.83:0/712496358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d81586b0 msgr2=0x7f13d8158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 --2- 172.21.15.83:0/712496358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d81586b0 0x7f13d8158aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f13d8072030 tx=0x7f13d000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:12.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 -- 172.21.15.83:0/712496358 shutdown_connections 2023-11-28T11:10:12.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 --2- 172.21.15.83:0/712496358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d81586b0 0x7f13d8158aa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 --2- 172.21.15.83:0/712496358 >> v2:172.21.15.90:3300/0 conn(0x7f13d8071710 0x7f13d80cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 -- 172.21.15.83:0/712496358 >> 172.21.15.83:0/712496358 conn(0x7f13d80cec90 msgr2=0x7f13d80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 -- 172.21.15.83:0/712496358 shutdown_connections 2023-11-28T11:10:12.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.380+0000 7f13dd597700 1 -- 172.21.15.83:0/712496358 wait complete. 2023-11-28T11:10:12.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.381+0000 7f13dd597700 1 Processor -- start 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.381+0000 7f13dd597700 1 -- start start 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 0x7f13d8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8071ae0 0x7f13d814b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13d81586b0 0x7f13d8156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8157dc0 con 0x7f13d8071710 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80b1ce0 con 0x7f13d8071ae0 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80b1fc0 con 0x7f13d81586b0 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 0x7f13d8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 0x7f13d8151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51258/0 (socket says 172.21.15.83:51258) 2023-11-28T11:10:12.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 -- 172.21.15.83:0/260465062 learned_addr learned my addr 172.21.15.83:0/260465062 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d67fc700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8071ae0 0x7f13d814b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d77fe700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13d81586b0 0x7f13d8156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 -- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13d81586b0 msgr2=0x7f13d8156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13d81586b0 0x7f13d8156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 -- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8071ae0 msgr2=0x7f13d814b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8071ae0 0x7f13d814b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 -- 172.21.15.83:0/260465062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009b30 con 0x7f13d8071710 2023-11-28T11:10:12.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 0x7f13d8151fd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f13c800d800 tx=0x7f13c800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80125f0 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8012750 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8158fe0 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8012b00 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.382+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d806dae0 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.383+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d80c24a0 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.384+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13c8012c60 con 0x7f13d8071710 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.384+0000 7f13bf7fe700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13c40682f0 0x7f13c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.384+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f13c80862f0 con 0x7f13d8071710 2023-11-28T11:10:12.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.384+0000 7f13d67fc700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13c40682f0 0x7f13c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.385+0000 7f13d67fc700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13c40682f0 0x7f13c406a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f13d81f13b0 tx=0x7f13cc007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.389+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13c8052a10 con 0x7f13d8071710 2023-11-28T11:10:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.573+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13d8074320 con 0x7f13d8071710 2023-11-28T11:10:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.575+0000 7f13bf7fe700 1 -- 172.21.15.83:0/260465062 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18022 (secure 0 0 0) 0x7f13c8056030 con 0x7f13d8071710 2023-11-28T11:10:12.576 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:12.577 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:47.730175+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.576+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13c40682f0 msgr2=0x7f13c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.576+0000 7f13dd597700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13c40682f0 0x7f13c406a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f13d81f13b0 tx=0x7f13cc007c90 comp rx=0 tx=0).stop 2023-11-28T11:10:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.576+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 msgr2=0x7f13d8151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.576+0000 7f13dd597700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 0x7f13d8151fd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f13c800d800 tx=0x7f13c800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 shutdown_connections 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13d81586b0 0x7f13d8156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13c40682f0 0x7f13c406a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8071710 0x7f13d8151fd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 --2- 172.21.15.83:0/260465062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8071ae0 0x7f13d814b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 >> 172.21.15.83:0/260465062 conn(0x7f13d80cec90 msgr2=0x7f13d806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 shutdown_connections 2023-11-28T11:10:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.577+0000 7f13dd597700 1 -- 172.21.15.83:0/260465062 wait complete. 2023-11-28T11:10:12.591 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:10:12.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.708+0000 7f6039b2b700 1 Processor -- start 2023-11-28T11:10:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6039b2b700 1 -- start start 2023-11-28T11:10:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6039b2b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f603406a950 0x7f603415ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6039b2b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60340b4e40 0x7f60340c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6039b2b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603415b160 con 0x7f60340ca070 2023-11-28T11:10:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6039b2b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603415b440 con 0x7f60340b4e40 2023-11-28T11:10:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6039b2b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603415b720 con 0x7f603406a950 2023-11-28T11:10:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6032ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60340b4e40 0x7f60340c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6033fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f60340ca070 0x7f6034071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49130/0 (socket says 172.21.15.83:49130) 2023-11-28T11:10:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.709+0000 7f6033fff700 1 -- 172.21.15.83:0/2493949213 learned_addr learned my addr 172.21.15.83:0/2493949213 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f60337fe700 1 --2- 172.21.15.83:0/2493949213 >> v2:172.21.15.90:3300/0 conn(0x7f603406a950 0x7f603415ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f6032ffd700 1 -- 172.21.15.83:0/2493949213 >> v2:172.21.15.90:3300/0 conn(0x7f603406a950 msgr2=0x7f603415ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f6032ffd700 1 --2- 172.21.15.83:0/2493949213 >> v2:172.21.15.90:3300/0 conn(0x7f603406a950 0x7f603415ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f6032ffd700 1 -- 172.21.15.83:0/2493949213 >> v1:172.21.15.83:6789/0 conn(0x7f60340ca070 legacy=0x7f6034071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f60327fc700 1 -- 172.21.15.83:0/2493949213 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259981519 0 0) 0x7f603415b160 con 0x7f60340ca070 2023-11-28T11:10:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f6032ffd700 1 -- 172.21.15.83:0/2493949213 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034071450 con 0x7f60340b4e40 2023-11-28T11:10:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f60337fe700 1 --2- 172.21.15.83:0/2493949213 >> v2:172.21.15.90:3300/0 conn(0x7f603406a950 0x7f603415ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f6032ffd700 1 --2- 172.21.15.83:0/2493949213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340c9b30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f603406e800 tx=0x7f6024009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67424549429e1443 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f60327fc700 1 -- 172.21.15.83:0/2493949213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024011830 con 0x7f60340b4e40 2023-11-28T11:10:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f60327fc700 1 -- 172.21.15.83:0/2493949213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6024011990 con 0x7f60340b4e40 2023-11-28T11:10:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.710+0000 7f60327fc700 1 -- 172.21.15.83:0/2493949213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024011d20 con 0x7f60340b4e40 2023-11-28T11:10:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 -- 172.21.15.83:0/2493949213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 msgr2=0x7f60340c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2493949213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340c9b30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f603406e800 tx=0x7f6024009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 -- 172.21.15.83:0/2493949213 shutdown_connections 2023-11-28T11:10:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2493949213 >> v2:172.21.15.90:3300/0 conn(0x7f603406a950 0x7f603415ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2493949213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340c9b30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 -- 172.21.15.83:0/2493949213 >> 172.21.15.83:0/2493949213 conn(0x7f60340cf4e0 msgr2=0x7f60340cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 -- 172.21.15.83:0/2493949213 shutdown_connections 2023-11-28T11:10:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.711+0000 7f6039b2b700 1 -- 172.21.15.83:0/2493949213 wait complete. 2023-11-28T11:10:12.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.712+0000 7f6039b2b700 1 Processor -- start 2023-11-28T11:10:12.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.712+0000 7f6039b2b700 1 -- start start 2023-11-28T11:10:12.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603406a950 0x7f60340c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f60340ca070 0x7f6034159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340c3c80 con 0x7f60340b4e40 2023-11-28T11:10:12.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340c30b0 con 0x7f603406a950 2023-11-28T11:10:12.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340c3350 con 0x7f60340ca070 2023-11-28T11:10:12.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51274/0 (socket says 172.21.15.83:51274) 2023-11-28T11:10:12.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 -- 172.21.15.83:0/2872640125 learned_addr learned my addr 172.21.15.83:0/2872640125 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:12.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6033fff700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f60340ca070 0x7f6034159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f60337fe700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603406a950 0x7f60340c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 -- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f60340ca070 msgr2=0x7f6034159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f60340ca070 0x7f6034159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 -- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603406a950 msgr2=0x7f60340c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603406a950 0x7f60340c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 -- 172.21.15.83:0/2872640125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240094a0 con 0x7f60340b4e40 2023-11-28T11:10:12.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6032ffd700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340adff0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6024000c00 tx=0x7f6024004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240038a0 con 0x7f60340b4e40 2023-11-28T11:10:12.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602401f070 con 0x7f60340b4e40 2023-11-28T11:10:12.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024003c40 con 0x7f60340b4e40 2023-11-28T11:10:12.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.713+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603406ad20 con 0x7f60340b4e40 2023-11-28T11:10:12.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.714+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603415ba00 con 0x7f60340b4e40 2023-11-28T11:10:12.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.714+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603406aeb0 con 0x7f60340b4e40 2023-11-28T11:10:12.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.715+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f602400f050 con 0x7f60340b4e40 2023-11-28T11:10:12.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.715+0000 7f601bfff700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f60200682f0 0x7f602006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.715+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6024021070 con 0x7f60340b4e40 2023-11-28T11:10:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.715+0000 7f60337fe700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f60200682f0 0x7f602006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.716+0000 7f60337fe700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f60200682f0 0x7f602006a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6028006fd0 tx=0x7f6028008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.719+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6024053f70 con 0x7f60340b4e40 2023-11-28T11:10:12.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.947+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6034159020 con 0x7f60340b4e40 2023-11-28T11:10:12.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.947+0000 7f601bfff700 1 -- 172.21.15.83:0/2872640125 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f6024015090 con 0x7f60340b4e40 2023-11-28T11:10:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:12.949 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.949+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f60200682f0 msgr2=0x7f602006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.949+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f60200682f0 0x7f602006a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6028006fd0 tx=0x7f6028008040 comp rx=0 tx=0).stop 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.949+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 msgr2=0x7f60340adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.949+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340adff0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6024000c00 tx=0x7f6024004750 comp rx=0 tx=0).stop 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 shutdown_connections 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f60340ca070 0x7f6034159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f603406a950 0x7f60340c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f60200682f0 0x7f602006a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 --2- 172.21.15.83:0/2872640125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60340b4e40 0x7f60340adff0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 >> 172.21.15.83:0/2872640125 conn(0x7f60340cf4e0 msgr2=0x7f60340b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 shutdown_connections 2023-11-28T11:10:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:12.950+0000 7f6039b2b700 1 -- 172.21.15.83:0/2872640125 wait complete. 2023-11-28T11:10:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:10:12.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.082+0000 7f58ae131700 1 Processor -- start 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.082+0000 7f58ae131700 1 -- start start 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58ae131700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 0x7f58a80cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58ae131700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f58a80b18a0 0x7f58a80b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58ae131700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a806ca60 con 0x7f58a806c5f0 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58ae131700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a815d480 con 0x7f58a80b18a0 2023-11-28T11:10:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58ae131700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80b2260 con 0x7f58a8159830 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a6ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f58a806c5f0 0x7f58a815de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49140/0 (socket says 172.21.15.83:49140) 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a6ffd700 1 -- 172.21.15.83:0/3057438694 learned_addr learned my addr 172.21.15.83:0/3057438694 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901975970 0 0) 0x7f58a806ca60 con 0x7f58a806c5f0 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a7fff700 1 --2- 172.21.15.83:0/3057438694 >> v2:172.21.15.90:3300/0 conn(0x7f58a80b18a0 0x7f58a80b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a77fe700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 0x7f58a80cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58900035b0 con 0x7f58a806c5f0 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 41975632 0 0) 0x7f58900035b0 con 0x7f58a806c5f0 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 msgr2=0x7f58a80cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 0x7f58a80cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 >> v2:172.21.15.90:3300/0 conn(0x7f58a80b18a0 msgr2=0x7f58a80b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 --2- 172.21.15.83:0/3057438694 >> v2:172.21.15.90:3300/0 conn(0x7f58a80b18a0 0x7f58a80b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80cf790 con 0x7f58a806c5f0 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f5898001670 con 0x7f58a806c5f0 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 >> v1:172.21.15.83:6789/0 conn(0x7f58a806c5f0 legacy=0x7f58a815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f5890003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.083+0000 7f58a67fc700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 0x7f5890004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5890004d90 0x7f5890005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a815d480 con 0x7f5890003a00 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80b2260 con 0x7f5890004d90 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589000ba10 con 0x7f58a8159830 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f5890003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a6ffd700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 0x7f5890004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a7fff700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5890004d90 0x7f5890005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 -- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 msgr2=0x7f5890004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a8159830 0x7f5890004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 -- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5890004d90 msgr2=0x7f5890005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5890004d90 0x7f5890005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 -- 172.21.15.83:0/3057438694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900035b0 con 0x7f5890003a00 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a7fff700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5890004d90 0x7f5890005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a77fe700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f5890003e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f589c009b60 tx=0x7f5890005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5464b7dfef6a744 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00e660 con 0x7f5890003a00 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00e7c0 con 0x7f5890003a00 2023-11-28T11:10:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.084+0000 7f58a67fc700 1 -- 172.21.15.83:0/3057438694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00eb30 con 0x7f5890003a00 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58ae131700 1 -- 172.21.15.83:0/3057438694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 msgr2=0x7f5890003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58ae131700 1 --2- 172.21.15.83:0/3057438694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f5890003e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f589c009b60 tx=0x7f5890005880 comp rx=0 tx=0).stop 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58a7fff700 1 -- 172.21.15.83:0/3057438694 reap_dead start 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58ae131700 1 -- 172.21.15.83:0/3057438694 shutdown_connections 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58ae131700 1 -- 172.21.15.83:0/3057438694 >> 172.21.15.83:0/3057438694 conn(0x7f58a806e9a0 msgr2=0x7f58a80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58ae131700 1 -- 172.21.15.83:0/3057438694 shutdown_connections 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.085+0000 7f58ae131700 1 -- 172.21.15.83:0/3057438694 wait complete. 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.086+0000 7f58ae131700 1 Processor -- start 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.086+0000 7f58ae131700 1 -- start start 2023-11-28T11:10:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58ae131700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f58a8155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58ae131700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58a814fa90 0x7f58a80c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58ae131700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a80c5a30 0x7f58a8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58ae131700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8150360 con 0x7f5890003a00 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58ae131700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80c3b80 con 0x7f58a814fa90 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58ae131700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a815bc60 con 0x7f58a80c5a30 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f58a8155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f58a8155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51292/0 (socket says 172.21.15.83:51292) 2023-11-28T11:10:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 -- 172.21.15.83:0/294960690 learned_addr learned my addr 172.21.15.83:0/294960690 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a7fff700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a80c5a30 0x7f58a8154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a6ffd700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58a814fa90 0x7f58a80c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 -- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a80c5a30 msgr2=0x7f58a8154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a80c5a30 0x7f58a8154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 -- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58a814fa90 msgr2=0x7f58a80c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58a814fa90 0x7f58a80c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 -- 172.21.15.83:0/294960690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a815d690 con 0x7f5890003a00 2023-11-28T11:10:13.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f58a77fe700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f58a8155670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f58900046d0 tx=0x7f589c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.087+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c021070 con 0x7f5890003a00 2023-11-28T11:10:13.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.088+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c004470 con 0x7f5890003a00 2023-11-28T11:10:13.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.088+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a80cfa70 con 0x7f5890003a00 2023-11-28T11:10:13.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.088+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00e660 con 0x7f5890003a00 2023-11-28T11:10:13.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.088+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a815d970 con 0x7f5890003a00 2023-11-28T11:10:13.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.089+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8074320 con 0x7f5890003a00 2023-11-28T11:10:13.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.089+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f589c02e030 con 0x7f5890003a00 2023-11-28T11:10:13.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.089+0000 7f588ffff700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f58940682f0 0x7f589406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.090+0000 7f58a6ffd700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f58940682f0 0x7f589406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.090+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f589c087040 con 0x7f5890003a00 2023-11-28T11:10:13.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.090+0000 7f58a6ffd700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f58940682f0 0x7f589406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58a8158e50 tx=0x7f5898007980 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.094+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f589c053760 con 0x7f5890003a00 2023-11-28T11:10:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.314+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a80c3950 con 0x7f5890003a00 2023-11-28T11:10:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.315+0000 7f588ffff700 1 -- 172.21.15.83:0/294960690 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4639 (secure 0 0 0) 0x7f589c056d80 con 0x7f5890003a00 2023-11-28T11:10:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:46.728367+0000","modified":"2023-11-28T11:09:49.740814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5098":{"gid":5098,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/686672772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":686672772},{"type":"v1","addr":"172.21.15.90:6833","nonce":686672772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.316+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f58940682f0 msgr2=0x7f589406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.316+0000 7f58ae131700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f58940682f0 0x7f589406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58a8158e50 tx=0x7f5898007980 comp rx=0 tx=0).stop 2023-11-28T11:10:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.316+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 msgr2=0x7f58a8155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.316+0000 7f58ae131700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f58a8155670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f58900046d0 tx=0x7f589c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.316+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 shutdown_connections 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58a80c5a30 0x7f58a8154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f58940682f0 0x7f589406a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5890003a00 0x7f58a8155670 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 --2- 172.21.15.83:0/294960690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58a814fa90 0x7f58a80c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 >> 172.21.15.83:0/294960690 conn(0x7f58a806e9a0 msgr2=0x7f58a806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 shutdown_connections 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.317+0000 7f58ae131700 1 -- 172.21.15.83:0/294960690 wait complete. 2023-11-28T11:10:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-28T11:10:13.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.452+0000 7fe6083d2700 1 Processor -- start 2023-11-28T11:10:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.452+0000 7fe6083d2700 1 -- start start 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe6083d2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe600159830 0x7fe6000cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe6083d2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe60006c5f0 0x7fe60015de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe6083d2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60006ca60 con 0x7fe6000b18a0 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe6083d2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60015d480 con 0x7fe60006c5f0 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe6083d2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60015e3d0 con 0x7fe600159830 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60596d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe60006c5f0 0x7fe60015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60696f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe6000b18a0 0x7fe6000cd9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49162/0 (socket says 172.21.15.83:49162) 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60696f700 1 -- 172.21.15.83:0/1010793108 learned_addr learned my addr 172.21.15.83:0/1010793108 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60616e700 1 --2- 172.21.15.83:0/1010793108 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe600159830 0x7fe6000cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60596d700 1 -- 172.21.15.83:0/1010793108 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe600159830 msgr2=0x7fe6000cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60596d700 1 --2- 172.21.15.83:0/1010793108 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe600159830 0x7fe6000cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60596d700 1 -- 172.21.15.83:0/1010793108 >> v1:172.21.15.83:6789/0 conn(0x7fe6000b18a0 legacy=0x7fe6000cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.453+0000 7fe60596d700 1 -- 172.21.15.83:0/1010793108 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000cf790 con 0x7fe60006c5f0 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe60596d700 1 --2- 172.21.15.83:0/1010793108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe60006c5f0 0x7fe60015de90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009840 tx=0x7fe5fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64122c1139cf1ece server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe60516c700 1 -- 172.21.15.83:0/1010793108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc011830 con 0x7fe60006c5f0 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe60516c700 1 -- 172.21.15.83:0/1010793108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc011990 con 0x7fe60006c5f0 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe60516c700 1 -- 172.21.15.83:0/1010793108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc011d20 con 0x7fe60006c5f0 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 -- 172.21.15.83:0/1010793108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe60006c5f0 msgr2=0x7fe60015de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 --2- 172.21.15.83:0/1010793108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe60006c5f0 0x7fe60015de90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009840 tx=0x7fe5fc009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 -- 172.21.15.83:0/1010793108 shutdown_connections 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 --2- 172.21.15.83:0/1010793108 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe600159830 0x7fe6000cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 --2- 172.21.15.83:0/1010793108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe60006c5f0 0x7fe60015de90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 -- 172.21.15.83:0/1010793108 >> 172.21.15.83:0/1010793108 conn(0x7fe60006e9a0 msgr2=0x7fe6000cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 -- 172.21.15.83:0/1010793108 shutdown_connections 2023-11-28T11:10:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.454+0000 7fe6083d2700 1 -- 172.21.15.83:0/1010793108 wait complete. 2023-11-28T11:10:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.455+0000 7fe6083d2700 1 Processor -- start 2023-11-28T11:10:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 -- start start 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe60006c5f0 0x7fe6001f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 0x7fe60006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe600159830 0x7fe60006b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001f36a0 con 0x7fe6000b18a0 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000c7c20 con 0x7fe600159830 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe6083d2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000c75c0 con 0x7fe60006c5f0 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe60596d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 0x7fe60006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe60596d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 0x7fe60006ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51308/0 (socket says 172.21.15.83:51308) 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe60596d700 1 -- 172.21.15.83:0/3241040498 learned_addr learned my addr 172.21.15.83:0/3241040498 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe60616e700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe60006c5f0 0x7fe6001f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.456+0000 7fe60696f700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe600159830 0x7fe60006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe60596d700 1 -- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe60006c5f0 msgr2=0x7fe6001f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe60596d700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe60006c5f0 0x7fe6001f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe60596d700 1 -- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe600159830 msgr2=0x7fe60006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe60596d700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe600159830 0x7fe60006b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe60596d700 1 -- 172.21.15.83:0/3241040498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0094a0 con 0x7fe6000b18a0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe60596d700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 0x7fe60006ab00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc000c00 tx=0x7fe5fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0036a0 con 0x7fe6000b18a0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc01e070 con 0x7fe6000b18a0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60015d690 con 0x7fe6000b18a0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc003a40 con 0x7fe6000b18a0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.457+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6000cfa70 con 0x7fe6000b18a0 2023-11-28T11:10:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.458+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60015d970 con 0x7fe6000b18a0 2023-11-28T11:10:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.458+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe5fc00f050 con 0x7fe6000b18a0 2023-11-28T11:10:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.459+0000 7fe5f6ffd700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe5e40682f0 0x7fe5e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.459+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe5fc086e20 con 0x7fe6000b18a0 2023-11-28T11:10:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.459+0000 7fe60616e700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe5e40682f0 0x7fe5e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.459+0000 7fe60616e700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe5e40682f0 0x7fe5e406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006b20 tx=0x7fe5f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.463+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5fc0534c0 con 0x7fe6000b18a0 2023-11-28T11:10:13.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.646+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe60006fef0 con 0x7fe6000b18a0 2023-11-28T11:10:13.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.648+0000 7fe5f6ffd700 1 -- 172.21.15.83:0/3241040498 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fe5fc020020 con 0x7fe6000b18a0 2023-11-28T11:10:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:13.649 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:09:47.730175+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:44.997179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:45.687674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:13.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe5e40682f0 msgr2=0x7fe5e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe5e40682f0 0x7fe5e406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006b20 tx=0x7fe5f0008040 comp rx=0 tx=0).stop 2023-11-28T11:10:13.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 msgr2=0x7fe60006ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 0x7fe60006ab00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc000c00 tx=0x7fe5fc004750 comp rx=0 tx=0).stop 2023-11-28T11:10:13.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 shutdown_connections 2023-11-28T11:10:13.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe600159830 0x7fe60006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe60006c5f0 0x7fe6001f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe5e40682f0 0x7fe5e406a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 --2- 172.21.15.83:0/3241040498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6000b18a0 0x7fe60006ab00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 >> 172.21.15.83:0/3241040498 conn(0x7fe60006e9a0 msgr2=0x7fe6000b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 shutdown_connections 2023-11-28T11:10:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.650+0000 7fe6083d2700 1 -- 172.21.15.83:0/3241040498 wait complete. 2023-11-28T11:10:13.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:10:13.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.784+0000 7f6c6a76b700 1 Processor -- start 2023-11-28T11:10:13.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.784+0000 7f6c6a76b700 1 -- start start 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c6a76b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c640cb3d0 0x7f6c640b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c6a76b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6406cad0 con 0x7f6c6415cd30 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c6a76b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c640af4a0 con 0x7f6c640b1d80 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c6a76b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6406bec0 con 0x7f6c640cb3d0 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c637fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c640cb3d0 0x7f6c640b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c63fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6c6415cd30 0x7f6c640cacc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49174/0 (socket says 172.21.15.83:49174) 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c637fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c640cb3d0 0x7f6c640b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51316/0 (socket says 172.21.15.83:51316) 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c63fff700 1 -- 172.21.15.83:0/4266899018 learned_addr learned my addr 172.21.15.83:0/4266899018 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c637fe700 1 -- 172.21.15.83:0/4266899018 >> v1:172.21.15.90:6789/0 conn(0x7f6c640b1d80 legacy=0x7f6c640cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c637fe700 1 -- 172.21.15.83:0/4266899018 >> v1:172.21.15.83:6789/0 conn(0x7f6c6415cd30 legacy=0x7f6c640cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c637fe700 1 -- 172.21.15.83:0/4266899018 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6406c1a0 con 0x7f6c640cb3d0 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c637fe700 1 --2- 172.21.15.83:0/4266899018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640b1840 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009840 tx=0x7f6c54009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2acc95811063b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c62ffd700 1 -- 172.21.15.83:0/4266899018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54004230 con 0x7f6c640cb3d0 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c62ffd700 1 -- 172.21.15.83:0/4266899018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c54004390 con 0x7f6c640cb3d0 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.785+0000 7f6c62ffd700 1 -- 172.21.15.83:0/4266899018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540036a0 con 0x7f6c640cb3d0 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4266899018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 msgr2=0x7f6c640b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4266899018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640b1840 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009840 tx=0x7f6c54009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4266899018 shutdown_connections 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4266899018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640b1840 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4266899018 >> 172.21.15.83:0/4266899018 conn(0x7f6c640cc690 msgr2=0x7f6c640cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4266899018 shutdown_connections 2023-11-28T11:10:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.786+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4266899018 wait complete. 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.788+0000 7f6c6a76b700 1 Processor -- start 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.788+0000 7f6c6a76b700 1 -- start start 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c640b1d80 0x7f6c6414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c6415cd30 0x7f6c640adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c3a60 con 0x7f6c640cb3d0 2023-11-28T11:10:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6414e5d0 con 0x7f6c640b1d80 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6415f3e0 con 0x7f6c6415cd30 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51322/0 (socket says 172.21.15.83:51322) 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 -- 172.21.15.83:0/4204563259 learned_addr learned my addr 172.21.15.83:0/4204563259 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 -- 172.21.15.83:0/4204563259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c6415cd30 msgr2=0x7f6c640adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c6415cd30 0x7f6c640adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 -- 172.21.15.83:0/4204563259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c640b1d80 msgr2=0x7f6c6414e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c640b1d80 0x7f6c6414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 -- 172.21.15.83:0/4204563259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540094a0 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c637fe700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640ada80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009b90 tx=0x7f6c54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54011630 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5401e070 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6406c760 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540119d0 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.789+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6406c480 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.790+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6406cf60 con 0x7f6c640cb3d0 2023-11-28T11:10:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.791+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c54017020 con 0x7f6c640cb3d0 2023-11-28T11:10:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.791+0000 7f6c60ff9700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c500682f0 0x7f6c5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.791+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6c54087110 con 0x7f6c640cb3d0 2023-11-28T11:10:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.791+0000 7f6c63fff700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c500682f0 0x7f6c5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.791+0000 7f6c63fff700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c500682f0 0x7f6c5006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c58000f60 tx=0x7f6c58007b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:13.795+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c54053830 con 0x7f6c640cb3d0 2023-11-28T11:10:14.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.023+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6c640742b0 con 0x7f6c640cb3d0 2023-11-28T11:10:14.857 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-28T11:10:14.856+0000 7fdf5c1f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:10:14.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.867+0000 7f6c60ff9700 1 -- 172.21.15.83:0/4204563259 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6c54056e50 con 0x7f6c640cb3d0 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c500682f0 msgr2=0x7f6c5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c500682f0 0x7f6c5006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c58000f60 tx=0x7f6c58007b10 comp rx=0 tx=0).stop 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 msgr2=0x7f6c640ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640ada80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009b90 tx=0x7f6c54004750 comp rx=0 tx=0).stop 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 shutdown_connections 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c6415cd30 0x7f6c640adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c640b1d80 0x7f6c6414e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c500682f0 0x7f6c5006a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 --2- 172.21.15.83:0/4204563259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c640cb3d0 0x7f6c640ada80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 >> 172.21.15.83:0/4204563259 conn(0x7f6c640cc690 msgr2=0x7f6c640af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 shutdown_connections 2023-11-28T11:10:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:14.870+0000 7f6c6a76b700 1 -- 172.21.15.83:0/4204563259 wait complete. 2023-11-28T11:10:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:10:14.874 INFO:tasks.ceph.mds.d.smithi090.stderr: -7284> 2023-11-28T11:09:50.136+0000 7f02c72f5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:10:14.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:10:14.908 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2023-11-28T11:10:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 Processor -- start 2023-11-28T11:10:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 -- start start 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f55a815e5c0 0x7f55a815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a8151a30 con 0x7f55a815b3b0 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80cb460 con 0x7f55a815e5c0 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.014+0000 7f55aed01700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80cb740 con 0x7f55a815b850 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55ad29e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55aca9d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f55a815e5c0 0x7f55a815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a7fff700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f55a815b3b0 0x7f55a80d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:47536/0 (socket says 172.21.15.83:47536) 2023-11-28T11:10:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a7fff700 1 -- 172.21.15.83:0/3961169045 learned_addr learned my addr 172.21.15.83:0/3961169045 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a77fe700 1 -- 172.21.15.83:0/3961169045 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536426591 0 0) 0x7f55a8151a30 con 0x7f55a815b3b0 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a77fe700 1 -- 172.21.15.83:0/3961169045 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55980035b0 con 0x7f55a815b3b0 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55ad29e700 1 -- 172.21.15.83:0/3961169045 >> v2:172.21.15.90:3300/0 conn(0x7f55a815e5c0 msgr2=0x7f55a815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55ad29e700 1 --2- 172.21.15.83:0/3961169045 >> v2:172.21.15.90:3300/0 conn(0x7f55a815e5c0 0x7f55a815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55ad29e700 1 -- 172.21.15.83:0/3961169045 >> v1:172.21.15.90:6789/0 conn(0x7f55a815b3b0 legacy=0x7f55a80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55ad29e700 1 -- 172.21.15.83:0/3961169045 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a815abb0 con 0x7f55a815b850 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55aca9d700 1 --2- 172.21.15.83:0/3961169045 >> v2:172.21.15.90:3300/0 conn(0x7f55a815e5c0 0x7f55a815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55ad29e700 1 --2- 172.21.15.83:0/3961169045 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a80ce1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009d90 tx=0x7f55a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ec75888a7f7f35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a77fe700 1 -- 172.21.15.83:0/3961169045 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000e070 con 0x7f55a815b850 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a77fe700 1 -- 172.21.15.83:0/3961169045 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a000bb20 con 0x7f55a815b850 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.015+0000 7f55a77fe700 1 -- 172.21.15.83:0/3961169045 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0012430 con 0x7f55a815b850 2023-11-28T11:10:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 -- 172.21.15.83:0/3961169045 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 msgr2=0x7f55a80ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 --2- 172.21.15.83:0/3961169045 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a80ce1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009d90 tx=0x7f55a000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 -- 172.21.15.83:0/3961169045 shutdown_connections 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 --2- 172.21.15.83:0/3961169045 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a80ce1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 --2- 172.21.15.83:0/3961169045 >> v2:172.21.15.90:3300/0 conn(0x7f55a815e5c0 0x7f55a815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 -- 172.21.15.83:0/3961169045 >> 172.21.15.83:0/3961169045 conn(0x7f55a806ea50 msgr2=0x7f55a8070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 -- 172.21.15.83:0/3961169045 shutdown_connections 2023-11-28T11:10:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.016+0000 7f55aed01700 1 -- 172.21.15.83:0/3961169045 wait complete. 2023-11-28T11:10:15.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.017+0000 7f55aed01700 1 Processor -- start 2023-11-28T11:10:15.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.017+0000 7f55aed01700 1 -- start start 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aed01700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aed01700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 0x7f55a806bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aed01700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55a806c150 0x7f55a806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aed01700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80b5f90 con 0x7f55a815e5c0 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aed01700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80b53e0 con 0x7f55a806c150 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aed01700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80b5660 con 0x7f55a815b850 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 0x7f55a806bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 0x7f55a806bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51336/0 (socket says 172.21.15.83:51336) 2023-11-28T11:10:15.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55a7fff700 1 -- 172.21.15.83:0/487435451 learned_addr learned my addr 172.21.15.83:0/487435451 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55aca9d700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.018+0000 7f55ad29e700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55a806c150 0x7f55a806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a7fff700 1 -- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 msgr2=0x7f55a806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a7fff700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a806d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a7fff700 1 -- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55a806c150 msgr2=0x7f55a806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a7fff700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55a806c150 0x7f55a806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a7fff700 1 -- 172.21.15.83:0/487435451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0094a0 con 0x7f55a815e5c0 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a7fff700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 0x7f55a806bc10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f559400c4d0 tx=0x7f559400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559400cea0 con 0x7f55a815e5c0 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5594005d70 con 0x7f55a815e5c0 2023-11-28T11:10:15.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a0009a70 con 0x7f55a815e5c0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940072f0 con 0x7f55a815e5c0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.019+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a815eed0 con 0x7f55a815e5c0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.020+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a815ae90 con 0x7f55a815e5c0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.020+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5594019030 con 0x7f55a815e5c0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.020+0000 7f55a57fa700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f558c0682f0 0x7f558c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.021+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f5594085100 con 0x7f55a815e5c0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.021+0000 7f55aca9d700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f558c0682f0 0x7f558c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.021+0000 7f55aca9d700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f558c0682f0 0x7f558c06a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f55a80b0e00 tx=0x7f559c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.025+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5594051740 con 0x7f55a815e5c0 2023-11-28T11:10:15.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.253+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f55a80cba20 con 0x7f55a815e5c0 2023-11-28T11:10:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.868+0000 7f55a57fa700 1 -- 172.21.15.83:0/487435451 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5594054d60 con 0x7f55a815e5c0 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f558c0682f0 msgr2=0x7f558c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f558c0682f0 0x7f558c06a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f55a80b0e00 tx=0x7f559c009210 comp rx=0 tx=0).stop 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 msgr2=0x7f55a806bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 0x7f55a806bc10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f559400c4d0 tx=0x7f559400c7e0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 shutdown_connections 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55a806c150 0x7f55a806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f55a815b850 0x7f55a806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f558c0682f0 0x7f558c06a7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 --2- 172.21.15.83:0/487435451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a815e5c0 0x7f55a806bc10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 >> 172.21.15.83:0/487435451 conn(0x7f55a815b3b0 msgr2=0x7f55a80d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 shutdown_connections 2023-11-28T11:10:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:15.871+0000 7f55aed01700 1 -- 172.21.15.83:0/487435451 wait complete. 2023-11-28T11:10:15.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:10:16.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.016+0000 7f42564be700 1 Processor -- start 2023-11-28T11:10:16.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.016+0000 7f42564be700 1 -- start start 2023-11-28T11:10:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f42564be700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4250071a50 0x7f42500cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f42564be700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425015d130 con 0x7f42500c9a50 2023-11-28T11:10:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f42564be700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425015d410 con 0x7f4250156e80 2023-11-28T11:10:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f42564be700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425015d6f0 con 0x7f4250071a50 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f42500c9a50 0x7f4250071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49204/0 (socket says 172.21.15.83:49204) 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424f7fe700 1 -- 172.21.15.83:0/374588806 learned_addr learned my addr 172.21.15.83:0/374588806 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323600855 0 0) 0x7f425015d130 con 0x7f42500c9a50 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42380035b0 con 0x7f42500c9a50 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f4254a5b700 1 --2- 172.21.15.83:0/374588806 >> v2:172.21.15.90:3300/0 conn(0x7f4250071a50 0x7f42500cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1507563133 0 0) 0x7f42380035b0 con 0x7f42500c9a50 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 >> v2:172.21.15.90:3300/0 conn(0x7f4250071a50 msgr2=0x7f42500cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 --2- 172.21.15.83:0/374588806 >> v2:172.21.15.90:3300/0 conn(0x7f4250071a50 0x7f42500cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 >> v1:172.21.15.90:6789/0 conn(0x7f4250156e80 legacy=0x7f42500ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425015d9d0 con 0x7f42500c9a50 2023-11-28T11:10:16.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.017+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f4240001670 con 0x7f42500c9a50 2023-11-28T11:10:16.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 >> v1:172.21.15.83:6789/0 conn(0x7f42500c9a50 legacy=0x7f4250071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f4238003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4238004850 0x7f4238004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4238005200 0x7f423800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425015d6f0 con 0x7f4238003a00 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423800c110 con 0x7f4238004850 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423800c2d0 con 0x7f4238005200 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f4238003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424f7fe700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4238004850 0x7f4238004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f4254a5b700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4238005200 0x7f423800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 -- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4238005200 msgr2=0x7f423800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4238005200 0x7f423800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 -- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4238004850 msgr2=0x7f4238004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4238004850 0x7f4238004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 -- 172.21.15.83:0/374588806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42380035b0 con 0x7f4238003a00 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f4254a5b700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4238005200 0x7f423800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f4254a5b700 1 -- 172.21.15.83:0/374588806 reap_dead start 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424ffff700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f4238003e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f42440017c0 tx=0x7f4238005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb5860ac61172ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440052d0 con 0x7f4238003a00 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244005430 con 0x7f4238003a00 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.018+0000 7f424effd700 1 -- 172.21.15.83:0/374588806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440038c0 con 0x7f4238003a00 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 -- 172.21.15.83:0/374588806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 msgr2=0x7f4238003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f4238003e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f42440017c0 tx=0x7f4238005880 comp rx=0 tx=0).stop 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 -- 172.21.15.83:0/374588806 shutdown_connections 2023-11-28T11:10:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 --2- 172.21.15.83:0/374588806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f4238003e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 -- 172.21.15.83:0/374588806 >> 172.21.15.83:0/374588806 conn(0x7f42500cf300 msgr2=0x7f42500cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 -- 172.21.15.83:0/374588806 shutdown_connections 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.019+0000 7f42564be700 1 -- 172.21.15.83:0/374588806 wait complete. 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.020+0000 7f42564be700 1 Processor -- start 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.020+0000 7f42564be700 1 -- start start 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f42564be700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f42500b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f42564be700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f425014e870 0x7f425014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f42564be700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f425014f510 0x7f425014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f42564be700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42500b5200 con 0x7f4238003a00 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f42564be700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42500c9510 con 0x7f425014e870 2023-11-28T11:10:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f42564be700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42501f2430 con 0x7f425014f510 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f42500b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f42500b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51354/0 (socket says 172.21.15.83:51354) 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 -- 172.21.15.83:0/1867899082 learned_addr learned my addr 172.21.15.83:0/1867899082 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424f7fe700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f425014e870 0x7f425014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f4254a5b700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f425014f510 0x7f425014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 -- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f425014f510 msgr2=0x7f425014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f425014f510 0x7f425014f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 -- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f425014e870 msgr2=0x7f425014efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f425014e870 0x7f425014efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 -- 172.21.15.83:0/1867899082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425015cde0 con 0x7f4238003a00 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f4254a5b700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f425014f510 0x7f425014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.021+0000 7f424ffff700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f42500b4cc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4244015040 tx=0x7f4244001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.022+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244001290 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.022+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424401d480 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.022+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4250157250 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.022+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424401d830 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.022+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425015dcb0 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.023+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250067af0 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.023+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f424402f070 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.023+0000 7f424cff9700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f423c0682f0 0x7f423c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.023+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f4244086b10 con 0x7f4238003a00 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.023+0000 7f424f7fe700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f423c0682f0 0x7f423c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.024+0000 7f424f7fe700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f423c0682f0 0x7f423c06a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f42501f5940 tx=0x7f4240001b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.028+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42440531d0 con 0x7f4238003a00 2023-11-28T11:10:16.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.213+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f42500673b0 con 0x7f4238003a00 2023-11-28T11:10:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.929+0000 7f424cff9700 1 -- 172.21.15.83:0/1867899082 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7f42440170d0 con 0x7f4238003a00 2023-11-28T11:10:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f423c0682f0 msgr2=0x7f423c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f423c0682f0 0x7f423c06a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f42501f5940 tx=0x7f4240001b20 comp rx=0 tx=0).stop 2023-11-28T11:10:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 msgr2=0x7f42500b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f42500b4cc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4244015040 tx=0x7f4244001120 comp rx=0 tx=0).stop 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 shutdown_connections 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f425014f510 0x7f425014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f423c0682f0 0x7f423c06a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4238003a00 0x7f42500b4cc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 --2- 172.21.15.83:0/1867899082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f425014e870 0x7f425014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 >> 172.21.15.83:0/1867899082 conn(0x7f42500cf300 msgr2=0x7f4250152aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 shutdown_connections 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:16.933+0000 7f42564be700 1 -- 172.21.15.83:0/1867899082 wait complete. 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:10:16.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:10:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.075+0000 7f2778e0c700 1 Processor -- start 2023-11-28T11:10:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.075+0000 7f2778e0c700 1 -- start start 2023-11-28T11:10:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f2778e0c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f277415a930 0x7f277406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f2778e0c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27740cd1a0 0x7f27740b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f2778e0c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f27740b56f0 0x7f277415e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f2778e0c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27740b5b70 con 0x7f277415a930 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f2778e0c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277406b830 con 0x7f27740b56f0 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f2778e0c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277406bad0 con 0x7f27740cd1a0 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f277259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f277415a930 0x7f277406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f277259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f277415a930 0x7f277406b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51362/0 (socket says 172.21.15.83:51362) 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.076+0000 7f277259c700 1 -- 172.21.15.83:0/4276876923 learned_addr learned my addr 172.21.15.83:0/4276876923 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f2772d9d700 1 --2- 172.21.15.83:0/4276876923 >> v2:172.21.15.90:3300/0 conn(0x7f27740b56f0 0x7f277415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f2771d9b700 1 --2- 172.21.15.83:0/4276876923 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27740cd1a0 0x7f27740b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277259c700 1 -- 172.21.15.83:0/4276876923 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27740cd1a0 msgr2=0x7f27740b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277259c700 1 --2- 172.21.15.83:0/4276876923 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27740cd1a0 0x7f27740b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277259c700 1 -- 172.21.15.83:0/4276876923 >> v2:172.21.15.90:3300/0 conn(0x7f27740b56f0 msgr2=0x7f277415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277259c700 1 --2- 172.21.15.83:0/4276876923 >> v2:172.21.15.90:3300/0 conn(0x7f27740b56f0 0x7f277415e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277259c700 1 -- 172.21.15.83:0/4276876923 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277406bdb0 con 0x7f277415a930 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277259c700 1 --2- 172.21.15.83:0/4276876923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f277415a930 0x7f277406b2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27740b2800 tx=0x7f2768009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13649cf017a0c65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277159a700 1 -- 172.21.15.83:0/4276876923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276800b870 con 0x7f277415a930 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277159a700 1 -- 172.21.15.83:0/4276876923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276800b9d0 con 0x7f277415a930 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f277159a700 1 -- 172.21.15.83:0/4276876923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276800bd60 con 0x7f277415a930 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.077+0000 7f2778e0c700 1 -- 172.21.15.83:0/4276876923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f277415a930 msgr2=0x7f277406b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 --2- 172.21.15.83:0/4276876923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f277415a930 0x7f277406b2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27740b2800 tx=0x7f2768009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 -- 172.21.15.83:0/4276876923 shutdown_connections 2023-11-28T11:10:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 --2- 172.21.15.83:0/4276876923 >> v2:172.21.15.90:3300/0 conn(0x7f27740b56f0 0x7f277415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 --2- 172.21.15.83:0/4276876923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f277415a930 0x7f277406b2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 --2- 172.21.15.83:0/4276876923 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f27740cd1a0 0x7f27740b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 -- 172.21.15.83:0/4276876923 >> 172.21.15.83:0/4276876923 conn(0x7f27740cc690 msgr2=0x7f27740cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 -- 172.21.15.83:0/4276876923 shutdown_connections 2023-11-28T11:10:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.078+0000 7f2778e0c700 1 -- 172.21.15.83:0/4276876923 wait complete. 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.079+0000 7f2778e0c700 1 Processor -- start 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.079+0000 7f2778e0c700 1 -- start start 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27740b56f0 0x7f2774152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 0x7f27741509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f277415a930 0x7f277414f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27741521c0 con 0x7f27740cd1a0 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740c4320 con 0x7f27740b56f0 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27741506e0 con 0x7f277415a930 2023-11-28T11:10:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 0x7f27741509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 0x7f27741509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51370/0 (socket says 172.21.15.83:51370) 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 -- 172.21.15.83:0/793486674 learned_addr learned my addr 172.21.15.83:0/793486674 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2772d9d700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f277415a930 0x7f277414f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f277259c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27740b56f0 0x7f2774152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 -- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f277415a930 msgr2=0x7f277414f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f277415a930 0x7f277414f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 -- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27740b56f0 msgr2=0x7f2774152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27740b56f0 0x7f2774152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 -- 172.21.15.83:0/793486674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009550 con 0x7f27740cd1a0 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2771d9b700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 0x7f27741509f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f276400e810 tx=0x7f276400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400c070 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764013610 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27640139c0 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.080+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27740ca670 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.081+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27740ca390 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.082+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277415ad00 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.082+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2764023070 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.082+0000 7f2762ffd700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27540682f0 0x7f275406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.082+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f27640859d0 con 0x7f27740cd1a0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.082+0000 7f277259c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27540682f0 0x7f275406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:17.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.083+0000 7f277259c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27540682f0 0x7f275406a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c00 tx=0x7f2768008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:17.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.086+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27640521d0 con 0x7f27740cd1a0 2023-11-28T11:10:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.270+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f27740742b0 con 0x7f27740cd1a0 2023-11-28T11:10:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.940+0000 7f2762ffd700 1 -- 172.21.15.83:0/793486674 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7f276400f040 con 0x7f27740cd1a0 2023-11-28T11:10:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27540682f0 msgr2=0x7f275406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27540682f0 0x7f275406a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c00 tx=0x7f2768008030 comp rx=0 tx=0).stop 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 msgr2=0x7f27741509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 0x7f27741509f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f276400e810 tx=0x7f276400eb20 comp rx=0 tx=0).stop 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 shutdown_connections 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f277415a930 0x7f277414f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27740b56f0 0x7f2774152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f27540682f0 0x7f275406a7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 --2- 172.21.15.83:0/793486674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27740cd1a0 0x7f27741509f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 >> 172.21.15.83:0/793486674 conn(0x7f27740cc690 msgr2=0x7f27740ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 shutdown_connections 2023-11-28T11:10:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:17.943+0000 7f2778e0c700 1 -- 172.21.15.83:0/793486674 wait complete. 2023-11-28T11:10:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-28T11:10:17.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:10:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.085+0000 7f66c9970700 1 Processor -- start 2023-11-28T11:10:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.085+0000 7f66c9970700 1 -- start start 2023-11-28T11:10:18.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c9970700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66c40d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c9970700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40d1780 con 0x7f66c40d0cb0 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c9970700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c4153b50 con 0x7f66c40ceee0 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c9970700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4161140 con 0x7f66c4160990 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f66c40d0cb0 0x7f66c406c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49240/0 (socket says 172.21.15.83:49240) 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c37fe700 1 -- 172.21.15.83:0/4005265536 learned_addr learned my addr 172.21.15.83:0/4005265536 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431649374 0 0) 0x7f66c40d1780 con 0x7f66c40d0cb0 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66ac0035b0 con 0x7f66c40d0cb0 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c27fc700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66c40d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1406841270 0 0) 0x7f66ac0035b0 con 0x7f66c40d0cb0 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 msgr2=0x7f66c40d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66c40d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 >> v1:172.21.15.90:6789/0 conn(0x7f66c40ceee0 legacy=0x7f66c4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40cf2b0 con 0x7f66c40d0cb0 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f66bc0019f0 con 0x7f66c40d0cb0 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 >> v1:172.21.15.83:6789/0 conn(0x7f66c40d0cb0 legacy=0x7f66c406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66ac003a30 0x7f66ac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac004d90 0x7f66ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.086+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4161140 con 0x7f66ac003a30 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac00bac0 con 0x7f66ac004d90 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac00bc80 con 0x7f66c4160990 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66ac003a30 0x7f66ac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c27fc700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c37fe700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac004d90 0x7f66ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 -- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 msgr2=0x7f66ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 -- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac004d90 msgr2=0x7f66ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac004d90 0x7f66ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 -- 172.21.15.83:0/4005265536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0035b0 con 0x7f66ac003a30 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c27fc700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c4160990 0x7f66ac004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66ac003a30 0x7f66ac003e40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f66b80017c0 tx=0x7f66ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e1bc7cb8b63138 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80052d0 con 0x7f66ac003a30 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8005430 con 0x7f66ac003a30 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.087+0000 7f66c1ffb700 1 -- 172.21.15.83:0/4005265536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80038c0 con 0x7f66ac003a30 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c9970700 1 -- 172.21.15.83:0/4005265536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66ac003a30 msgr2=0x7f66ac003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c9970700 1 --2- 172.21.15.83:0/4005265536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66ac003a30 0x7f66ac003e40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f66b80017c0 tx=0x7f66ac005880 comp rx=0 tx=0).stop 2023-11-28T11:10:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c37fe700 1 -- 172.21.15.83:0/4005265536 reap_dead start 2023-11-28T11:10:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c9970700 1 -- 172.21.15.83:0/4005265536 shutdown_connections 2023-11-28T11:10:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c9970700 1 -- 172.21.15.83:0/4005265536 >> 172.21.15.83:0/4005265536 conn(0x7f66c406de20 msgr2=0x7f66c406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c9970700 1 -- 172.21.15.83:0/4005265536 shutdown_connections 2023-11-28T11:10:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.088+0000 7f66c9970700 1 -- 172.21.15.83:0/4005265536 wait complete. 2023-11-28T11:10:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.089+0000 7f66c9970700 1 Processor -- start 2023-11-28T11:10:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.089+0000 7f66c9970700 1 -- start start 2023-11-28T11:10:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac003a30 0x7f66c40b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c40b6940 0x7f66c40c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 0x7f66c40c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c414d3d0 con 0x7f66c40c5e60 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c414d8d0 con 0x7f66ac003a30 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40c7260 con 0x7f66c40b6940 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 0x7f66c40c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 0x7f66c40c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51392/0 (socket says 172.21.15.83:51392) 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 -- 172.21.15.83:0/2842590521 learned_addr learned my addr 172.21.15.83:0/2842590521 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac003a30 0x7f66c40b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c27fc700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c40b6940 0x7f66c40c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 -- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c40b6940 msgr2=0x7f66c40c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c40b6940 0x7f66c40c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 -- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac003a30 msgr2=0x7f66c40b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac003a30 0x7f66c40b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 -- 172.21.15.83:0/2842590521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c406bdb0 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c37fe700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 0x7f66c40c6300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f66c40b39d0 tx=0x7f66bc007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00eb50 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc00ecb0 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c40d14a0 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc013420 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.090+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c406c090 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.091+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c40742b0 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.092+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66bc013580 con 0x7f66c40c5e60 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.092+0000 7f66abfff700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b00682f0 0x7f66b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.092+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b00682f0 0x7f66b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.092+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f66bc086160 con 0x7f66c40c5e60 2023-11-28T11:10:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.093+0000 7f66c2ffd700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b00682f0 0x7f66b006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f66b8000c00 tx=0x7f66b8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.097+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66bc052b10 con 0x7f66c40c5e60 2023-11-28T11:10:18.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.292+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f66c406fca0 con 0x7f66c40c5e60 2023-11-28T11:10:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.297+0000 7f66abfff700 1 -- 172.21.15.83:0/2842590521 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f66bc056130 con 0x7f66c40c5e60 2023-11-28T11:10:18.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.299+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b00682f0 msgr2=0x7f66b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.299+0000 7f66c9970700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b00682f0 0x7f66b006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f66b8000c00 tx=0x7f66b8009030 comp rx=0 tx=0).stop 2023-11-28T11:10:18.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.299+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 msgr2=0x7f66c40c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.299+0000 7f66c9970700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 0x7f66c40c6300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f66c40b39d0 tx=0x7f66bc007c90 comp rx=0 tx=0).stop 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 shutdown_connections 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f66b00682f0 0x7f66b006a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c40c5e60 0x7f66c40c6300 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac003a30 0x7f66c40b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 --2- 172.21.15.83:0/2842590521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66c40b6940 0x7f66c40c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 >> 172.21.15.83:0/2842590521 conn(0x7f66c406de20 msgr2=0x7f66c40b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 shutdown_connections 2023-11-28T11:10:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.300+0000 7f66c9970700 1 -- 172.21.15.83:0/2842590521 wait complete. 2023-11-28T11:10:18.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:10:18.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.432+0000 7f14b8c3c700 1 Processor -- start 2023-11-28T11:10:18.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.432+0000 7f14b8c3c700 1 -- start start 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b8c3c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 0x7f14b406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b8c3c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b406c440 con 0x7f14b415aad0 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b8c3c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b406b490 con 0x7f14b406b8b0 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b8c3c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cb220 con 0x7f14b406ab00 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14b415aad0 0x7f14b40b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49256/0 (socket says 172.21.15.83:49256) 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b259c700 1 -- 172.21.15.83:0/1406887871 learned_addr learned my addr 172.21.15.83:0/1406887871 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b1d9b700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 0x7f14b406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773096397 0 0) 0x7f14b406c440 con 0x7f14b415aad0 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f149c0035b0 con 0x7f14b415aad0 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3529087978 0 0) 0x7f149c0035b0 con 0x7f14b415aad0 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 msgr2=0x7f14b406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 0x7f14b406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 >> v1:172.21.15.90:6789/0 conn(0x7f14b406b8b0 legacy=0x7f14b406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b415aea0 con 0x7f14b415aad0 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f14a4001670 con 0x7f14b415aad0 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 >> v1:172.21.15.83:6789/0 conn(0x7f14b415aad0 legacy=0x7f14b40b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f149c003a30 0x7f149c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f149c004850 0x7f149c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 0x7f149c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.433+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cb220 con 0x7f149c004850 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c00bb10 con 0x7f149c003a30 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c00bcd0 con 0x7f14b406ab00 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f149c004850 0x7f149c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b259c700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f149c003a30 0x7f149c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b2d9d700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 0x7f149c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 -- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 msgr2=0x7f149c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14b406ab00 0x7f149c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 -- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f149c003a30 msgr2=0x7f149c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f149c003a30 0x7f149c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 -- 172.21.15.83:0/1406887871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0035b0 con 0x7f149c004850 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b259c700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f149c003a30 0x7f149c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b1d9b700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f149c004850 0x7f149c004cc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009b60 tx=0x7f149c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f131f665c98b3507 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800e6c0 con 0x7f149c004850 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800e820 con 0x7f149c004850 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.434+0000 7f14b159a700 1 -- 172.21.15.83:0/1406887871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800eb90 con 0x7f149c004850 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b8c3c700 1 -- 172.21.15.83:0/1406887871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f149c004850 msgr2=0x7f149c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/1406887871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f149c004850 0x7f149c004cc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009b60 tx=0x7f149c005880 comp rx=0 tx=0).stop 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b2d9d700 1 -- 172.21.15.83:0/1406887871 reap_dead start 2023-11-28T11:10:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b8c3c700 1 -- 172.21.15.83:0/1406887871 shutdown_connections 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b8c3c700 1 -- 172.21.15.83:0/1406887871 >> 172.21.15.83:0/1406887871 conn(0x7f14b406e9a0 msgr2=0x7f14b40cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b8c3c700 1 -- 172.21.15.83:0/1406887871 shutdown_connections 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.435+0000 7f14b8c3c700 1 -- 172.21.15.83:0/1406887871 wait complete. 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.436+0000 7f14b8c3c700 1 Processor -- start 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.436+0000 7f14b8c3c700 1 -- start start 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b8c3c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f149c004850 0x7f14b4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b8c3c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14b40b3ca0 0x7f14b40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b8c3c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 0x7f14b4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b8c3c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cfa90 con 0x7f14b4157f20 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b8c3c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40b4650 con 0x7f14b40b3ca0 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b8c3c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4158c10 con 0x7f149c004850 2023-11-28T11:10:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 0x7f14b4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 0x7f14b4158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51412/0 (socket says 172.21.15.83:51412) 2023-11-28T11:10:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 -- 172.21.15.83:0/3752901199 learned_addr learned my addr 172.21.15.83:0/3752901199 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b1d9b700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14b40b3ca0 0x7f14b40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b259c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f149c004850 0x7f14b4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 -- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f149c004850 msgr2=0x7f14b4153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f149c004850 0x7f14b4153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 -- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14b40b3ca0 msgr2=0x7f14b40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14b40b3ca0 0x7f14b40b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 -- 172.21.15.83:0/3752901199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40ca8f0 con 0x7f14b4157f20 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14b2d9d700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 0x7f14b4158390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b29d0 tx=0x7f14ac00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.437+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00eee0 con 0x7f14b4157f20 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.438+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac003c10 con 0x7f14b4157f20 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.438+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b40ca610 con 0x7f14b4157f20 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.438+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0050d0 con 0x7f14b4157f20 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.438+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b40cabd0 con 0x7f14b4157f20 2023-11-28T11:10:18.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.439+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b4074320 con 0x7f14b4157f20 2023-11-28T11:10:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.439+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f14ac005230 con 0x7f14b4157f20 2023-11-28T11:10:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.439+0000 7f14a2ffd700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14940682f0 0x7f149406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.439+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f14ac085900 con 0x7f14b4157f20 2023-11-28T11:10:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.439+0000 7f14b259c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14940682f0 0x7f149406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.440+0000 7f14b259c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14940682f0 0x7f149406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f14a4000c00 tx=0x7f14a4007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.443+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f14ac052320 con 0x7f14b4157f20 2023-11-28T11:10:18.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.698+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f14b406b490 con 0x7f14b4157f20 2023-11-28T11:10:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.940+0000 7f14a2ffd700 1 -- 172.21.15.83:0/3752901199 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v108) v1 ==== 142+0+0 (secure 0 0 0) 0x7f14b406b490 con 0x7f14b4157f20 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14940682f0 msgr2=0x7f149406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14940682f0 0x7f149406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f14a4000c00 tx=0x7f14a4007040 comp rx=0 tx=0).stop 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 msgr2=0x7f14b4158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 0x7f14b4158390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b29d0 tx=0x7f14ac00ed50 comp rx=0 tx=0).stop 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 shutdown_connections 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f14940682f0 0x7f149406a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b4157f20 0x7f14b4158390 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f149c004850 0x7f14b4153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 --2- 172.21.15.83:0/3752901199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14b40b3ca0 0x7f14b40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:18.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 >> 172.21.15.83:0/3752901199 conn(0x7f14b406e9a0 msgr2=0x7f14b40b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:18.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 shutdown_connections 2023-11-28T11:10:18.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:18.942+0000 7f14b8c3c700 1 -- 172.21.15.83:0/3752901199 wait complete. 2023-11-28T11:10:18.955 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T11:10:18.956 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T11:10:18.956 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:10:18.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:10:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.075+0000 7fbc4ae8c700 1 Processor -- start 2023-11-28T11:10:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.075+0000 7fbc4ae8c700 1 -- start start 2023-11-28T11:10:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc4ae8c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc44159830 0x7fbc440cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc4ae8c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc4406c5f0 0x7fbc4415de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc4ae8c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbc440b18a0 0x7fbc440b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc4ae8c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4415d480 con 0x7fbc4406c5f0 2023-11-28T11:10:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc4ae8c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc440b2260 con 0x7fbc440b18a0 2023-11-28T11:10:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc4ae8c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4406ca60 con 0x7fbc44159830 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc4406c5f0 0x7fbc4415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc4406c5f0 0x7fbc4415de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51418/0 (socket says 172.21.15.83:51418) 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 -- 172.21.15.83:0/3162974529 learned_addr learned my addr 172.21.15.83:0/3162974529 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc49429700 1 --2- 172.21.15.83:0/3162974529 >> v2:172.21.15.90:3300/0 conn(0x7fbc440b18a0 0x7fbc440b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc48c28700 1 --2- 172.21.15.83:0/3162974529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc44159830 0x7fbc440cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 -- 172.21.15.83:0/3162974529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc44159830 msgr2=0x7fbc440cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 --2- 172.21.15.83:0/3162974529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc44159830 0x7fbc440cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 -- 172.21.15.83:0/3162974529 >> v2:172.21.15.90:3300/0 conn(0x7fbc440b18a0 msgr2=0x7fbc440b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 --2- 172.21.15.83:0/3162974529 >> v2:172.21.15.90:3300/0 conn(0x7fbc440b18a0 0x7fbc440b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 -- 172.21.15.83:0/3162974529 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4415e3d0 con 0x7fbc4406c5f0 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc43fff700 1 --2- 172.21.15.83:0/3162974529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc4415de90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbc440cd410 tx=0x7fbc30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1991b4b973998600 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc437fe700 1 -- 172.21.15.83:0/3162974529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000b870 con 0x7fbc4406c5f0 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc437fe700 1 -- 172.21.15.83:0/3162974529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3000b9d0 con 0x7fbc4406c5f0 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.076+0000 7fbc437fe700 1 -- 172.21.15.83:0/3162974529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000bd60 con 0x7fbc4406c5f0 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3162974529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 msgr2=0x7fbc4415de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3162974529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc4415de90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbc440cd410 tx=0x7fbc30009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3162974529 shutdown_connections 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3162974529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc44159830 0x7fbc440cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3162974529 >> v2:172.21.15.90:3300/0 conn(0x7fbc440b18a0 0x7fbc440b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3162974529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc4415de90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3162974529 >> 172.21.15.83:0/3162974529 conn(0x7fbc4406e9a0 msgr2=0x7fbc440cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3162974529 shutdown_connections 2023-11-28T11:10:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.077+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3162974529 wait complete. 2023-11-28T11:10:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.079+0000 7fbc4ae8c700 1 Processor -- start 2023-11-28T11:10:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.079+0000 7fbc4ae8c700 1 -- start start 2023-11-28T11:10:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc440b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc440b18a0 0x7fbc4414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc44159830 0x7fbc440c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4414fee0 con 0x7fbc4406c5f0 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440c54d0 con 0x7fbc44159830 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4414f920 con 0x7fbc440b18a0 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc440b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc440b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51422/0 (socket says 172.21.15.83:51422) 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 -- 172.21.15.83:0/3748751909 learned_addr learned my addr 172.21.15.83:0/3748751909 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc43fff700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc440b18a0 0x7fbc4414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc49429700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc44159830 0x7fbc440c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 -- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc440b18a0 msgr2=0x7fbc4414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:19.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc440b18a0 0x7fbc4414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 -- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc44159830 msgr2=0x7fbc440c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc44159830 0x7fbc440c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 -- 172.21.15.83:0/3748751909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc30009550 con 0x7fbc4406c5f0 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc49429700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc44159830 0x7fbc440c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc48c28700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc440b43c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d830 tx=0x7fbc3800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380125f0 con 0x7fbc4406c5f0 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc38012750 con 0x7fbc4406c5f0 2023-11-28T11:10:19.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc440cfa70 con 0x7fbc4406c5f0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38012b00 con 0x7fbc4406c5f0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.080+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc440cf790 con 0x7fbc4406c5f0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.081+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4415d690 con 0x7fbc4406c5f0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.082+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc38012c60 con 0x7fbc4406c5f0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.082+0000 7fbc417fa700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbc280682f0 0x7fbc2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.082+0000 7fbc43fff700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbc280682f0 0x7fbc2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.082+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbc38085f90 con 0x7fbc4406c5f0 2023-11-28T11:10:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.083+0000 7fbc43fff700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbc280682f0 0x7fbc2806a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc30000c00 tx=0x7fbc30008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.086+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbc380529b0 con 0x7fbc4406c5f0 2023-11-28T11:10:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.339+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fbc44074320 con 0x7fbc4406c5f0 2023-11-28T11:10:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.946+0000 7fbc417fa700 1 -- 172.21.15.83:0/3748751909 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v109) v1 ==== 152+0+0 (secure 0 0 0) 0x7fbc38055fd0 con 0x7fbc4406c5f0 2023-11-28T11:10:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbc280682f0 msgr2=0x7fbc2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbc280682f0 0x7fbc2806a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc30000c00 tx=0x7fbc30008030 comp rx=0 tx=0).stop 2023-11-28T11:10:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 msgr2=0x7fbc440b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc440b43c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d830 tx=0x7fbc3800dbf0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 shutdown_connections 2023-11-28T11:10:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc44159830 0x7fbc440c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbc280682f0 0x7fbc2806a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4406c5f0 0x7fbc440b43c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 --2- 172.21.15.83:0/3748751909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbc440b18a0 0x7fbc4414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 >> 172.21.15.83:0/3748751909 conn(0x7fbc4406e9a0 msgr2=0x7fbc440b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 shutdown_connections 2023-11-28T11:10:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:19.949+0000 7fbc4ae8c700 1 -- 172.21.15.83:0/3748751909 wait complete. 2023-11-28T11:10:19.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:10:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.088+0000 7f2fad0fc700 1 Processor -- start 2023-11-28T11:10:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.088+0000 7f2fad0fc700 1 -- start start 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fad0fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa815cd30 0x7f2fa80cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fad0fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fa80cb200 0x7f2fa80b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fad0fc700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2fa80b1d80 0x7f2fa81606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fad0fc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa80cb670 con 0x7f2fa80cb200 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fad0fc700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa806cad0 con 0x7f2fa80b1d80 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fad0fc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80af4a0 con 0x7f2fa815cd30 2023-11-28T11:10:20.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fa80cb200 0x7f2fa80b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fa80cb200 0x7f2fa80b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51430/0 (socket says 172.21.15.83:51430) 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 -- 172.21.15.83:0/1740725468 learned_addr learned my addr 172.21.15.83:0/1740725468 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa6d9d700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa815cd30 0x7f2fa80cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa759e700 1 --2- 172.21.15.83:0/1740725468 >> v2:172.21.15.90:3300/0 conn(0x7f2fa80b1d80 0x7f2fa81606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 -- 172.21.15.83:0/1740725468 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa815cd30 msgr2=0x7f2fa80cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa815cd30 0x7f2fa80cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 -- 172.21.15.83:0/1740725468 >> v2:172.21.15.90:3300/0 conn(0x7f2fa80b1d80 msgr2=0x7f2fa81606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 --2- 172.21.15.83:0/1740725468 >> v2:172.21.15.90:3300/0 conn(0x7f2fa80b1d80 0x7f2fa81606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 -- 172.21.15.83:0/1740725468 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa806bec0 con 0x7f2fa80cb200 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa759e700 1 --2- 172.21.15.83:0/1740725468 >> v2:172.21.15.90:3300/0 conn(0x7f2fa80b1d80 0x7f2fa81606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa6d9d700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa815cd30 0x7f2fa80cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.089+0000 7f2fa659c700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80cb200 0x7f2fa80b1840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80cfad0 tx=0x7f2f98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4739ac811a28039 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fa5d9b700 1 -- 172.21.15.83:0/1740725468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800b870 con 0x7f2fa80cb200 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fa5d9b700 1 -- 172.21.15.83:0/1740725468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9800b9d0 con 0x7f2fa80cb200 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fa5d9b700 1 -- 172.21.15.83:0/1740725468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800bd60 con 0x7f2fa80cb200 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1740725468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80cb200 msgr2=0x7f2fa80b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80cb200 0x7f2fa80b1840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80cfad0 tx=0x7f2f98009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1740725468 shutdown_connections 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa815cd30 0x7f2fa80cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1740725468 >> v2:172.21.15.90:3300/0 conn(0x7f2fa80b1d80 0x7f2fa81606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1740725468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80cb200 0x7f2fa80b1840 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1740725468 >> 172.21.15.83:0/1740725468 conn(0x7f2fa80cc690 msgr2=0x7f2fa80cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1740725468 shutdown_connections 2023-11-28T11:10:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.090+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1740725468 wait complete. 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 Processor -- start 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 -- start start 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 0x7f2fa815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa80cb200 0x7f2fa81f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa815cd30 0x7f2fa80b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8152980 con 0x7f2fa80b1d80 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa815e9c0 con 0x7f2fa815cd30 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.092+0000 7f2fad0fc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80b44d0 con 0x7f2fa80cb200 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 0x7f2fa815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 0x7f2fa815dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51438/0 (socket says 172.21.15.83:51438) 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 -- 172.21.15.83:0/1267078399 learned_addr learned my addr 172.21.15.83:0/1267078399 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa759e700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa815cd30 0x7f2fa80b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa659c700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa80cb200 0x7f2fa81f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 -- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa80cb200 msgr2=0x7f2fa81f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa80cb200 0x7f2fa81f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 -- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa815cd30 msgr2=0x7f2fa80b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa815cd30 0x7f2fa80b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 -- 172.21.15.83:0/1267078399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98009550 con 0x7f2fa80b1d80 2023-11-28T11:10:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fa6d9d700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 0x7f2fa815dd30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00e840 tx=0x7f2f9c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004230 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c004390 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa806c480 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0036a0 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.093+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa806c1a0 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.094+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa806c760 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.094+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f9c022070 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.095+0000 7f2f8f7fe700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f940682f0 0x7f2f9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.095+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2f9c0855b0 con 0x7f2fa80b1d80 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.095+0000 7f2fa659c700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f940682f0 0x7f2f9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.095+0000 7f2fa659c700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f940682f0 0x7f2f9406a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f98014040 tx=0x7f2f98008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:20.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.099+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f9c051fd0 con 0x7f2fa80b1d80 2023-11-28T11:10:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.282+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fa815e790 con 0x7f2fa80b1d80 2023-11-28T11:10:20.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.286+0000 7f2f8f7fe700 1 -- 172.21.15.83:0/1267078399 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2fa815e790 con 0x7f2fa80b1d80 2023-11-28T11:10:20.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f940682f0 msgr2=0x7f2f9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f940682f0 0x7f2f9406a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f98014040 tx=0x7f2f98008030 comp rx=0 tx=0).stop 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 msgr2=0x7f2fa815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 0x7f2fa815dd30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00e840 tx=0x7f2f9c00ec00 comp rx=0 tx=0).stop 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 shutdown_connections 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa815cd30 0x7f2fa80b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2f940682f0 0x7f2f9406a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fa80b1d80 0x7f2fa815dd30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 --2- 172.21.15.83:0/1267078399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2fa80cb200 0x7f2fa81f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 >> 172.21.15.83:0/1267078399 conn(0x7f2fa80cc690 msgr2=0x7f2fa80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:20.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 shutdown_connections 2023-11-28T11:10:20.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.288+0000 7f2fad0fc700 1 -- 172.21.15.83:0/1267078399 wait complete. 2023-11-28T11:10:20.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:10:20.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:20.344 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:20.388 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:10:20.389 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:10:20.389 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:20.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:20.407 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:10:20.408 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:20.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:20.448 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:10:20.448 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:20.465 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:20.465 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:10:20.465 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:20.508 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:20.508 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:10:20.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.626+0000 7fb56f422700 1 Processor -- start 2023-11-28T11:10:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.626+0000 7fb56f422700 1 -- start start 2023-11-28T11:10:20.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56f422700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb56815a510 0x7fb5680cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56f422700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb5680cfdf0 0x7fb5680b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56f422700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56815f1d0 con 0x7fb56815a8e0 2023-11-28T11:10:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56f422700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5680cd780 con 0x7fb5680cfdf0 2023-11-28T11:10:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56f422700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56815eef0 con 0x7fb56815a510 2023-11-28T11:10:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56c9bd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb5680cfdf0 0x7fb5680b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56d9bf700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb56815a8e0 0x7fb5680b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49308/0 (socket says 172.21.15.83:49308) 2023-11-28T11:10:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56d9bf700 1 -- 172.21.15.83:0/3745273664 learned_addr learned my addr 172.21.15.83:0/3745273664 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56d1be700 1 --2- 172.21.15.83:0/3745273664 >> v2:172.21.15.90:3300/0 conn(0x7fb56815a510 0x7fb5680cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb55ffff700 1 -- 172.21.15.83:0/3745273664 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352134572 0 0) 0x7fb56815f1d0 con 0x7fb56815a8e0 2023-11-28T11:10:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56c9bd700 1 -- 172.21.15.83:0/3745273664 >> v2:172.21.15.90:3300/0 conn(0x7fb56815a510 msgr2=0x7fb5680cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56c9bd700 1 --2- 172.21.15.83:0/3745273664 >> v2:172.21.15.90:3300/0 conn(0x7fb56815a510 0x7fb5680cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56c9bd700 1 -- 172.21.15.83:0/3745273664 >> v1:172.21.15.83:6789/0 conn(0x7fb56815a8e0 legacy=0x7fb5680b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.627+0000 7fb56c9bd700 1 -- 172.21.15.83:0/3745273664 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5680cedd0 con 0x7fb5680cfdf0 2023-11-28T11:10:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56c9bd700 1 --2- 172.21.15.83:0/3745273664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb5680b2960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb5680ce6b0 tx=0x7fb564009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40928765a5a55cac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb55ffff700 1 -- 172.21.15.83:0/3745273664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011830 con 0x7fb5680cfdf0 2023-11-28T11:10:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb55ffff700 1 -- 172.21.15.83:0/3745273664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564011990 con 0x7fb5680cfdf0 2023-11-28T11:10:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb55ffff700 1 -- 172.21.15.83:0/3745273664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011d20 con 0x7fb5680cfdf0 2023-11-28T11:10:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 -- 172.21.15.83:0/3745273664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 msgr2=0x7fb5680b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 --2- 172.21.15.83:0/3745273664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb5680b2960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb5680ce6b0 tx=0x7fb564009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 -- 172.21.15.83:0/3745273664 shutdown_connections 2023-11-28T11:10:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 --2- 172.21.15.83:0/3745273664 >> v2:172.21.15.90:3300/0 conn(0x7fb56815a510 0x7fb5680cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 --2- 172.21.15.83:0/3745273664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb5680b2960 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 -- 172.21.15.83:0/3745273664 >> 172.21.15.83:0/3745273664 conn(0x7fb5680cd960 msgr2=0x7fb5680cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 -- 172.21.15.83:0/3745273664 shutdown_connections 2023-11-28T11:10:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.628+0000 7fb56f422700 1 -- 172.21.15.83:0/3745273664 wait complete. 2023-11-28T11:10:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.629+0000 7fb56f422700 1 Processor -- start 2023-11-28T11:10:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 -- start start 2023-11-28T11:10:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb568146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb56815a510 0x7fb5680c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb56815a8e0 0x7fb5680c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb568153660 con 0x7fb5680cfdf0 2023-11-28T11:10:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681ef920 con 0x7fb56815a8e0 2023-11-28T11:10:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56f422700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680c6660 con 0x7fb56815a510 2023-11-28T11:10:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56d1be700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb568146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56d1be700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb568146460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51450/0 (socket says 172.21.15.83:51450) 2023-11-28T11:10:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.630+0000 7fb56d1be700 1 -- 172.21.15.83:0/657248895 learned_addr learned my addr 172.21.15.83:0/657248895 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d9bf700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb56815a8e0 0x7fb5680c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56c9bd700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb56815a510 0x7fb5680c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d1be700 1 -- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb56815a510 msgr2=0x7fb5680c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d1be700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb56815a510 0x7fb5680c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d1be700 1 -- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb56815a8e0 msgr2=0x7fb5680c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d1be700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb56815a8e0 0x7fb5680c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d1be700 1 -- 172.21.15.83:0/657248895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640094a0 con 0x7fb5680cfdf0 2023-11-28T11:10:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d9bf700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb56815a8e0 0x7fb5680c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56c9bd700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb56815a510 0x7fb5680c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56d1be700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb568146460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb55800e810 tx=0x7fb55800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558013610 con 0x7fb5680cfdf0 2023-11-28T11:10:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb558013770 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5680cf390 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558013b20 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.631+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5680cf0b0 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.632+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5680b4440 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.632+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb558023070 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.633+0000 7fb55dffb700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb54c0682f0 0x7fb54c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.633+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb5580857e0 con 0x7fb5680cfdf0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.633+0000 7fb56c9bd700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb54c0682f0 0x7fb54c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.633+0000 7fb56c9bd700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb54c0682f0 0x7fb54c06a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb568152470 tx=0x7fb564008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.637+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb558052180 con 0x7fb5680cfdf0 2023-11-28T11:10:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.861+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb568074320 con 0x7fb5680cfdf0 2023-11-28T11:10:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.862+0000 7fb55dffb700 1 -- 172.21.15.83:0/657248895 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fb558021de0 con 0x7fb5680cfdf0 2023-11-28T11:10:20.863 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:20.863 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/208849165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":208849165},{"type":"v1","addr":"172.21.15.83:6836","nonce":208849165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/390839321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":390839321},{"type":"v1","addr":"172.21.15.83:6837","nonce":390839321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3224227278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3224227278},{"type":"v1","addr":"172.21.15.90:6835","nonce":3224227278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2841159036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2841159036},{"type":"v1","addr":"172.21.15.90:6833","nonce":2841159036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-11-28T11:10:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb54c0682f0 msgr2=0x7fb54c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb54c0682f0 0x7fb54c06a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb568152470 tx=0x7fb564008030 comp rx=0 tx=0).stop 2023-11-28T11:10:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 msgr2=0x7fb568146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb568146460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb55800e810 tx=0x7fb55800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 shutdown_connections 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb56815a8e0 0x7fb5680c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb54c0682f0 0x7fb54c06a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5680cfdf0 0x7fb568146460 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 --2- 172.21.15.83:0/657248895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb56815a510 0x7fb5680c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 >> 172.21.15.83:0/657248895 conn(0x7fb5680cd960 msgr2=0x7fb5680d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 shutdown_connections 2023-11-28T11:10:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:20.864+0000 7fb56f422700 1 -- 172.21.15.83:0/657248895 wait complete. 2023-11-28T11:10:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 19 2023-11-28T11:10:20.877 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:10:20.877 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:10:20.878 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:10:20.878 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:20.878 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:10:20.878 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:10:20.879 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:10:20.879 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:20.879 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:10:20.879 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:10:20.879 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:10:20.879 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:20.879 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:10:20.879 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:10:20.880 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:10:20.880 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:20.887 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:10:20.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:10:20.888 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:10:20.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:10:20.890 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:10:20.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:10:20.891 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:10:20.892 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:10:20.929 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:10:20.930 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:20.930 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:10:20.931 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:20.933 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:10:20.933 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:20.933 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:10:20.934 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:20.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:10:21.008 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-28T11:10:21.008 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-28T11:10:21.021 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2023-11-28T11:10:21.021 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2023-11-28T11:10:21.022 INFO:tasks.ceph.mds.d.smithi090.stderr:2023-11-28T11:10:21.020+0000 7fc284311700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-28T11:10:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.063+0000 7fbf1f37a700 1 Processor -- start 2023-11-28T11:10:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.063+0000 7fbf1f37a700 1 -- start start 2023-11-28T11:10:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1f37a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf180cd040 0x7fbf18070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1f37a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180710f0 con 0x7fbf18070cb0 2023-11-28T11:10:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1f37a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180cb9f0 con 0x7fbf180b6480 2023-11-28T11:10:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1f37a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180b6850 con 0x7fbf180cd040 2023-11-28T11:10:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1c915700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf180cd040 0x7fbf18070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1d917700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbf18070cb0 0x7fbf180cf0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49346/0 (socket says 172.21.15.83:49346) 2023-11-28T11:10:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1c915700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf180cd040 0x7fbf18070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51484/0 (socket says 172.21.15.83:51484) 2023-11-28T11:10:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.064+0000 7fbf1d917700 1 -- 172.21.15.83:0/3781833394 learned_addr learned my addr 172.21.15.83:0/3781833394 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1c915700 1 -- 172.21.15.83:0/3781833394 >> v1:172.21.15.90:6789/0 conn(0x7fbf180b6480 legacy=0x7fbf180cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1c915700 1 -- 172.21.15.83:0/3781833394 >> v1:172.21.15.83:6789/0 conn(0x7fbf18070cb0 legacy=0x7fbf180cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf17fff700 1 -- 172.21.15.83:0/3781833394 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225657999 0 0) 0x7fbf180cb9f0 con 0x7fbf180b6480 2023-11-28T11:10:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1c915700 1 -- 172.21.15.83:0/3781833394 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180ce900 con 0x7fbf180cd040 2023-11-28T11:10:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1c915700 1 --2- 172.21.15.83:0/3781833394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf180cd040 0x7fbf18070770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009840 tx=0x7fbf0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4779f64e1278880 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf17fff700 1 -- 172.21.15.83:0/3781833394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c004230 con 0x7fbf180cd040 2023-11-28T11:10:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf17fff700 1 -- 172.21.15.83:0/3781833394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c004390 con 0x7fbf180cd040 2023-11-28T11:10:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf17fff700 1 -- 172.21.15.83:0/3781833394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0036a0 con 0x7fbf180cd040 2023-11-28T11:10:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3781833394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf180cd040 msgr2=0x7fbf18070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3781833394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf180cd040 0x7fbf18070770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009840 tx=0x7fbf0c009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3781833394 shutdown_connections 2023-11-28T11:10:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3781833394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf180cd040 0x7fbf18070770 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.065+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3781833394 >> 172.21.15.83:0/3781833394 conn(0x7fbf180d0c90 msgr2=0x7fbf180d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.066+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3781833394 shutdown_connections 2023-11-28T11:10:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.066+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3781833394 wait complete. 2023-11-28T11:10:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.067+0000 7fbf1f37a700 1 Processor -- start 2023-11-28T11:10:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.067+0000 7fbf1f37a700 1 -- start start 2023-11-28T11:10:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1f37a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 0x7fbf181502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1f37a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf180b6480 0x7fbf181549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1f37a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf18154ee0 0x7fbf180c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1f37a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181f3d80 con 0x7fbf18070cb0 2023-11-28T11:10:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1f37a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181f4210 con 0x7fbf180b6480 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1f37a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181f3450 con 0x7fbf18154ee0 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 0x7fbf181502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 0x7fbf181502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51488/0 (socket says 172.21.15.83:51488) 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 -- 172.21.15.83:0/3895595668 learned_addr learned my addr 172.21.15.83:0/3895595668 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1c915700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf180b6480 0x7fbf181549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d917700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf18154ee0 0x7fbf180c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 -- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf18154ee0 msgr2=0x7fbf180c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf18154ee0 0x7fbf180c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 -- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf180b6480 msgr2=0x7fbf181549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf180b6480 0x7fbf181549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d116700 1 -- 172.21.15.83:0/3895595668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0094a0 con 0x7fbf18070cb0 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.068+0000 7fbf1d917700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf18154ee0 0x7fbf180c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.069+0000 7fbf1d116700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 0x7fbf181502b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c8c0 tx=0x7fbf1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.069+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10003e30 con 0x7fbf18070cb0 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.069+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf10005070 con 0x7fbf18070cb0 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.069+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180b1230 con 0x7fbf18070cb0 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.069+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10005420 con 0x7fbf18070cb0 2023-11-28T11:10:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.069+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180b0f50 con 0x7fbf18070cb0 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.070+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf180ceba0 con 0x7fbf18070cb0 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.070+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf10010070 con 0x7fbf18070cb0 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.070+0000 7fbf15ffb700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf040682f0 0x7fbf0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.071+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbf10085730 con 0x7fbf18070cb0 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.071+0000 7fbf1c915700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf040682f0 0x7fbf0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.071+0000 7fbf1c915700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf040682f0 0x7fbf0406a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0090b0 tx=0x7fbf0c013040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.075+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf10052150 con 0x7fbf18070cb0 2023-11-28T11:10:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:21.272+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbf18074240 con 0x7fbf18070cb0 2023-11-28T11:10:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.052+0000 7fbf15ffb700 1 -- 172.21.15.83:0/3895595668 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf10055770 con 0x7fbf18070cb0 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf040682f0 msgr2=0x7fbf0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf040682f0 0x7fbf0406a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0090b0 tx=0x7fbf0c013040 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 msgr2=0x7fbf181502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 0x7fbf181502b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c8c0 tx=0x7fbf1000cc80 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 shutdown_connections 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbf18154ee0 0x7fbf180c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf040682f0 0x7fbf0406a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf18070cb0 0x7fbf181502b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 --2- 172.21.15.83:0/3895595668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf180b6480 0x7fbf181549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 >> 172.21.15.83:0/3895595668 conn(0x7fbf180d0c90 msgr2=0x7fbf180b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 shutdown_connections 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.054+0000 7fbf1f37a700 1 -- 172.21.15.83:0/3895595668 wait complete. 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-28T11:10:22.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:10:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.186+0000 7ffaa5e6c700 1 Processor -- start 2023-11-28T11:10:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.186+0000 7ffaa5e6c700 1 -- start start 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffaa5e6c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 0x7ffaa00cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffaa5e6c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00ce620 con 0x7ffaa00b1e30 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffaa5e6c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00ce8c0 con 0x7ffaa0161020 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffaa5e6c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00ceba0 con 0x7ffaa00af030 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffa9ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffaa00b1e30 0x7ffaa006b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49364/0 (socket says 172.21.15.83:49364) 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffa9ffff700 1 -- 172.21.15.83:0/798550993 learned_addr learned my addr 172.21.15.83:0/798550993 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210948377 0 0) 0x7ffaa00ce620 con 0x7ffaa00b1e30 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffa9f7fe700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 0x7ffaa00cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa880035b0 con 0x7ffaa00b1e30 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.187+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4093046363 0 0) 0x7ffa880035b0 con 0x7ffaa00b1e30 2023-11-28T11:10:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 msgr2=0x7ffaa00cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 0x7ffaa00cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 >> v1:172.21.15.90:6789/0 conn(0x7ffaa0161020 legacy=0x7ffaa00b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00cee80 con 0x7ffaa00b1e30 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7ffa980019f0 con 0x7ffaa00b1e30 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 >> v1:172.21.15.83:6789/0 conn(0x7ffaa00b1e30 legacy=0x7ffaa006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa88003a30 0x7ffa88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 0x7ffa88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa88004d90 0x7ffa88005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00ceba0 con 0x7ffa88004d90 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8800ba10 con 0x7ffa88003a30 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8800bba0 con 0x7ffaa00af030 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9f7fe700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa88003a30 0x7ffa88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa88004d90 0x7ffa88005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9effd700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 0x7ffa88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 -- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 msgr2=0x7ffa88004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa00af030 0x7ffa88004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 -- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa88003a30 msgr2=0x7ffa88003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa88003a30 0x7ffa88003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 -- 172.21.15.83:0/798550993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880035b0 con 0x7ffa88004d90 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9f7fe700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa88003a30 0x7ffa88003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa88004d90 0x7ffa88005200 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800a810 tx=0x7ffa88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca288559abf6b839 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98012600 con 0x7ffa88004d90 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98012780 con 0x7ffa88004d90 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.188+0000 7ffa9e7fc700 1 -- 172.21.15.83:0/798550993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98012b30 con 0x7ffa88004d90 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/798550993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa88004d90 msgr2=0x7ffa88005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/798550993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa88004d90 0x7ffa88005200 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800a810 tx=0x7ffa88005880 comp rx=0 tx=0).stop 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffa9ffff700 1 -- 172.21.15.83:0/798550993 reap_dead start 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/798550993 shutdown_connections 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/798550993 >> 172.21.15.83:0/798550993 conn(0x7ffaa00cc720 msgr2=0x7ffaa00ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/798550993 shutdown_connections 2023-11-28T11:10:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.189+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/798550993 wait complete. 2023-11-28T11:10:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.190+0000 7ffaa5e6c700 1 Processor -- start 2023-11-28T11:10:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.190+0000 7ffaa5e6c700 1 -- start start 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffaa5e6c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaa01ef870 0x7ffaa01f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffaa5e6c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa01f2a00 0x7ffaa00ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffaa5e6c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 0x7ffaa00ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffaa5e6c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa01f34d0 con 0x7ffaa01efc40 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffaa5e6c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00ac850 con 0x7ffaa01ef870 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffaa5e6c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa01f2f10 con 0x7ffaa01f2a00 2023-11-28T11:10:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 0x7ffaa00ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 0x7ffaa00ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51516/0 (socket says 172.21.15.83:51516) 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 -- 172.21.15.83:0/3814906379 learned_addr learned my addr 172.21.15.83:0/3814906379 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9effd700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa01f2a00 0x7ffaa00ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9f7fe700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaa01ef870 0x7ffaa01f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 -- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa01f2a00 msgr2=0x7ffaa00ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa01f2a00 0x7ffaa00ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 -- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaa01ef870 msgr2=0x7ffaa01f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaa01ef870 0x7ffaa01f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.191+0000 7ffa9ffff700 1 -- 172.21.15.83:0/3814906379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00af400 con 0x7ffaa01efc40 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.192+0000 7ffa9ffff700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 0x7ffaa00ad830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ffa98012000 tx=0x7ffaa00ce7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.192+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98025070 con 0x7ffaa01efc40 2023-11-28T11:10:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.192+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9800a280 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.192+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa00cf160 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.192+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98012d40 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.192+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa00cf680 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.193+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa0074320 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.193+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffa98010070 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.193+0000 7ffa87fff700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ffa8c0682f0 0x7ffa8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.194+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ffa98087130 con 0x7ffaa01efc40 2023-11-28T11:10:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.194+0000 7ffa9f7fe700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ffa8c0682f0 0x7ffa8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.194+0000 7ffa9f7fe700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ffa8c0682f0 0x7ffa8c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009ba0 tx=0x7ffa94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.198+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffa98053b20 con 0x7ffaa01efc40 2023-11-28T11:10:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.452+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffaa00b5c20 con 0x7ffaa01efc40 2023-11-28T11:10:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.452+0000 7ffa87fff700 1 -- 172.21.15.83:0/3814906379 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5298 (secure 0 0 0) 0x7ffa98057140 con 0x7ffaa01efc40 2023-11-28T11:10:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:22.454 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:10:22.454 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVly5GSMRAArlawRsTlzQ8deoXsPnYmsg==", 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.455 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVl2bkZAhAA15xpS3O6vPGdnCqW98I/0w==", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlCEbQOBAA8FKGXdCurT68yNjSUEJigQ==", 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.457 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlQFxUCRAA9BmlOX7cM5ewNextI+L6lg==", 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-28T11:10:22.458 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAAymVlMHf8JRAAqxuxtuaslP93VXhK17KfXA==", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQADymVlUrMkJRAAxCUtzDNzfHRTJ4jLzeTa0Q==", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.459 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAGymVlZqxOJBAAzhcB9ERpDVy5FCZNn1uMLg==", 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAJymVlPZEhJBAAm8fhc9fRDDq3ZROV4Mez1Q==", 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAPymVlqwREKBAAsRlS/3ztcuWSZ9nRXZDKuA==", 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQASymVlQKsoKBAABR2d0tZ/TLRKIDyQrAKVkA==", 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.462 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-28T11:10:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAZymVlqduLAhAAPau0X+UZigxTAoWpmUnzRg==", 2023-11-28T11:10:22.463 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAcymVlckzLBBAAH4vnm0MWED/F40ZWq2uIog==", 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVllJ2YDRAAE9HeK1zyWWnOIRh09fAG3Q==", 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:10:22.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVln8eKFhAAPGbMQ2n6PsaH6hEURGRJvw==", 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlXJRCExAATH4e/FL5+nvYhdsZ7mB9LQ==", 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlo7c6HBAAHpELpEY29uzkqcJ+jAaxRA==", 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-28T11:10:22.468 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlC9hgFRAAL7ZVG6EzaROjM+to9qi5ZQ==", 2023-11-28T11:10:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:10:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:10:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:10:22.469 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlBfMxLhAA6N4Cnno8jNhgzS5eYJKWqQ==", 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:22.470 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlHSGFKhAA3pr4QAikKKmqCfS0orTyWw==", 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:22.472 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-28T11:10:22.472 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:10:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.454+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ffa8c0682f0 msgr2=0x7ffa8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.454+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ffa8c0682f0 0x7ffa8c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009ba0 tx=0x7ffa94008040 comp rx=0 tx=0).stop 2023-11-28T11:10:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.454+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 msgr2=0x7ffaa00ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.454+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 0x7ffaa00ad830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ffa98012000 tx=0x7ffaa00ce7b0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.454+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 shutdown_connections 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ffa8c0682f0 0x7ffa8c06a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa01efc40 0x7ffaa00ad830 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaa01ef870 0x7ffaa01f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 --2- 172.21.15.83:0/3814906379 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffaa01f2a00 0x7ffaa00ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 >> 172.21.15.83:0/3814906379 conn(0x7ffaa00cc720 msgr2=0x7ffaa015b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 shutdown_connections 2023-11-28T11:10:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.455+0000 7ffaa5e6c700 1 -- 172.21.15.83:0/3814906379 wait complete. 2023-11-28T11:10:22.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:22.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.593+0000 7f62dcca3700 1 Processor -- start 2023-11-28T11:10:22.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.593+0000 7f62dcca3700 1 -- start start 2023-11-28T11:10:22.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.593+0000 7f62dcca3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 0x7f62d8167ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.593+0000 7f62dcca3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d815cd80 con 0x7f62d80c9ac0 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62dcca3700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d815d040 con 0x7f62d806d500 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62dcca3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d815d320 con 0x7f62d81656f0 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d5d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62d80c9ac0 0x7f62d80c9f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49380/0 (socket says 172.21.15.83:49380) 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d5d9b700 1 -- 172.21.15.83:0/48873743 learned_addr learned my addr 172.21.15.83:0/48873743 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d6d9d700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 0x7f62d8167ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725505948 0 0) 0x7f62d815cd80 con 0x7f62d80c9ac0 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c00035b0 con 0x7f62d80c9ac0 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241250151 0 0) 0x7f62d815d040 con 0x7f62d806d500 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d815cd80 con 0x7f62d806d500 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208478980 0 0) 0x7f62c00035b0 con 0x7f62d80c9ac0 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 msgr2=0x7f62d8167ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 0x7f62d8167ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 >> v1:172.21.15.90:6789/0 conn(0x7f62d806d500 legacy=0x7f62d806d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d815d600 con 0x7f62d80c9ac0 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f62c8001670 con 0x7f62d80c9ac0 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 >> v1:172.21.15.83:6789/0 conn(0x7f62d80c9ac0 legacy=0x7f62d80c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 0x7f62c0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c0004d90 0x7f62c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d815d320 con 0x7f62c0004120 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c000bc00 con 0x7f62c0004d90 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c000bdc0 con 0x7f62d81656f0 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d659c700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 0x7f62c0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.594+0000 7f62d6d9d700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c0004d90 0x7f62c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d5d9b700 1 -- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 msgr2=0x7f62c0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81656f0 0x7f62c0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d5d9b700 1 -- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c0004d90 msgr2=0x7f62c0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62c0004d90 0x7f62c0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d5d9b700 1 -- 172.21.15.83:0/48873743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c00035b0 con 0x7f62c0004120 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62c0004850 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62c8001eb0 tx=0x7f62c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766c4b3a25ce2e7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800cd20 con 0x7f62c0004120 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8005960 con 0x7f62c0004120 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d559a700 1 -- 172.21.15.83:0/48873743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800cea0 con 0x7f62c0004120 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62dcca3700 1 -- 172.21.15.83:0/48873743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 msgr2=0x7f62c0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62dcca3700 1 --2- 172.21.15.83:0/48873743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62c0004850 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62c8001eb0 tx=0x7f62c0005880 comp rx=0 tx=0).stop 2023-11-28T11:10:22.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62d6d9d700 1 -- 172.21.15.83:0/48873743 reap_dead start 2023-11-28T11:10:22.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.595+0000 7f62dcca3700 1 -- 172.21.15.83:0/48873743 shutdown_connections 2023-11-28T11:10:22.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.596+0000 7f62dcca3700 1 -- 172.21.15.83:0/48873743 >> 172.21.15.83:0/48873743 conn(0x7f62d80cf470 msgr2=0x7f62d80cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.596+0000 7f62dcca3700 1 -- 172.21.15.83:0/48873743 shutdown_connections 2023-11-28T11:10:22.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.596+0000 7f62dcca3700 1 -- 172.21.15.83:0/48873743 wait complete. 2023-11-28T11:10:22.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.597+0000 7f62dcca3700 1 Processor -- start 2023-11-28T11:10:22.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.597+0000 7f62dcca3700 1 -- start start 2023-11-28T11:10:22.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62d806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d806b360 0x7f62d8146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81468c0 0x7f62d80ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b5260 con 0x7f62c0004120 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d814fc50 con 0x7f62d806b360 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d806b7d0 con 0x7f62d81468c0 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62d806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62d806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51530/0 (socket says 172.21.15.83:51530) 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 -- 172.21.15.83:0/3276940548 learned_addr learned my addr 172.21.15.83:0/3276940548 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d806b360 0x7f62d8146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d6d9d700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81468c0 0x7f62d80ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 -- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81468c0 msgr2=0x7f62d80ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81468c0 0x7f62d80ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 -- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d806b360 msgr2=0x7f62d8146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d806b360 0x7f62d8146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 -- 172.21.15.83:0/3276940548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80cd040 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62d659c700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62d806ae20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00f800 tx=0x7f62cc00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0052e0 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc005440 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d815d8e0 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0038c0 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.598+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d8157010 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.599+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d80ab8b0 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.600+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62cc003a20 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.600+0000 7f62c6ffd700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62b80682f0 0x7f62b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.600+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f62cc0858d0 con 0x7f62c0004120 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.600+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62b80682f0 0x7f62b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.601+0000 7f62d5d9b700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62b80682f0 0x7f62b806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f62c8000c00 tx=0x7f62c8007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.604+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62d80ab8b0 con 0x7f62c0004120 2023-11-28T11:10:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.787+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62d80ab8b0 con 0x7f62c0004120 2023-11-28T11:10:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.788+0000 7f62c6ffd700 1 -- 172.21.15.83:0/3276940548 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14366 (secure 0 0 0) 0x7f62d80ab8b0 con 0x7f62c0004120 2023-11-28T11:10:22.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:22.790 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:22.042389+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62b80682f0 msgr2=0x7f62b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62b80682f0 0x7f62b806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f62c8000c00 tx=0x7f62c8007f90 comp rx=0 tx=0).stop 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 msgr2=0x7f62d806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62d806ae20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00f800 tx=0x7f62cc00fb10 comp rx=0 tx=0).stop 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 shutdown_connections 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f62d81468c0 0x7f62d80ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f62b80682f0 0x7f62b806a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c0004120 0x7f62d806ae20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 --2- 172.21.15.83:0/3276940548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d806b360 0x7f62d8146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 >> 172.21.15.83:0/3276940548 conn(0x7f62d80cf470 msgr2=0x7f62d8071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 shutdown_connections 2023-11-28T11:10:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.791+0000 7f62dcca3700 1 -- 172.21.15.83:0/3276940548 wait complete. 2023-11-28T11:10:22.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:22.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:22.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.927+0000 7f3b13950700 1 Processor -- start 2023-11-28T11:10:22.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.927+0000 7f3b13950700 1 -- start start 2023-11-28T11:10:22.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b13950700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c06e370 0x7f3b0c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b13950700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b0c0b4bb0 0x7f3b0c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b13950700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c06d4a0 con 0x7f3b0c15c890 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b13950700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b5050 con 0x7f3b0c0b4bb0 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b13950700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b5ae0 con 0x7f3b0c06e370 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b11eed700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b0c0b4bb0 0x7f3b0c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b11eed700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b0c0b4bb0 0x7f3b0c06b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51542/0 (socket says 172.21.15.83:51542) 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b11eed700 1 -- 172.21.15.83:0/824636570 learned_addr learned my addr 172.21.15.83:0/824636570 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.928+0000 7f3b10eeb700 1 --2- 172.21.15.83:0/824636570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c06e370 0x7f3b0c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b11eed700 1 -- 172.21.15.83:0/824636570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c06e370 msgr2=0x7f3b0c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b11eed700 1 --2- 172.21.15.83:0/824636570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c06e370 0x7f3b0c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b11eed700 1 -- 172.21.15.83:0/824636570 >> v1:172.21.15.90:6789/0 conn(0x7f3b0c15c890 legacy=0x7f3b0c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b11eed700 1 -- 172.21.15.83:0/824636570 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c06c400 con 0x7f3b0c0b4bb0 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b11eed700 1 --2- 172.21.15.83:0/824636570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c06b150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009e50 tx=0x7f3b0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ff2497697ddb6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b03fff700 1 -- 172.21.15.83:0/824636570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400e070 con 0x7f3b0c0b4bb0 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b03fff700 1 -- 172.21.15.83:0/824636570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0400b910 con 0x7f3b0c0b4bb0 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b03fff700 1 -- 172.21.15.83:0/824636570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400bcc0 con 0x7f3b0c0b4bb0 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b13950700 1 -- 172.21.15.83:0/824636570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c0b4bb0 msgr2=0x7f3b0c06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b13950700 1 --2- 172.21.15.83:0/824636570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c06b150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009e50 tx=0x7f3b0400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:22.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b13950700 1 -- 172.21.15.83:0/824636570 shutdown_connections 2023-11-28T11:10:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b13950700 1 --2- 172.21.15.83:0/824636570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c06b150 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b13950700 1 --2- 172.21.15.83:0/824636570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c06e370 0x7f3b0c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.929+0000 7f3b13950700 1 -- 172.21.15.83:0/824636570 >> 172.21.15.83:0/824636570 conn(0x7f3b0c0b2b60 msgr2=0x7f3b0c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.930+0000 7f3b13950700 1 -- 172.21.15.83:0/824636570 shutdown_connections 2023-11-28T11:10:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.930+0000 7f3b13950700 1 -- 172.21.15.83:0/824636570 wait complete. 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 Processor -- start 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 -- start start 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 0x7f3b0c1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c15c890 0x7f3b0c0c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c6690 con 0x7f3b0c06e370 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c63b0 con 0x7f3b0c0b4bb0 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.931+0000 7f3b13950700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b2090 con 0x7f3b0c15c890 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 0x7f3b0c1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 0x7f3b0c1472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51544/0 (socket says 172.21.15.83:51544) 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 -- 172.21.15.83:0/3444801545 learned_addr learned my addr 172.21.15.83:0/3444801545 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 -- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c15c890 msgr2=0x7f3b0c0c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b10eeb700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b11eed700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c15c890 0x7f3b0c0c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c15c890 0x7f3b0c0c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 -- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b0c0b4bb0 msgr2=0x7f3b0c0c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c0c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 -- 172.21.15.83:0/3444801545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009b30 con 0x7f3b0c06e370 2023-11-28T11:10:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b116ec700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 0x7f3b0c1472f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c4d0 tx=0x7f3afc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0052d0 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc005430 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c0b5520 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0038f0 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.932+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c0b5240 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.933+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c0b5800 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.933+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3afc019030 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.934+0000 7f3b01ffb700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3af00682f0 0x7f3af006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.934+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3afc084d80 con 0x7f3b0c06e370 2023-11-28T11:10:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.934+0000 7f3b10eeb700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3af00682f0 0x7f3af006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.934+0000 7f3b10eeb700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3af00682f0 0x7f3af006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0c8f50 tx=0x7f3b08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:22.938+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3afc0517f0 con 0x7f3b0c06e370 2023-11-28T11:10:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.121+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3b0c0c7920 con 0x7f3b0c06e370 2023-11-28T11:10:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.121+0000 7f3b01ffb700 1 -- 172.21.15.83:0/3444801545 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3afc054e10 con 0x7f3b0c06e370 2023-11-28T11:10:23.123 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3af00682f0 msgr2=0x7f3af006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3af00682f0 0x7f3af006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0c8f50 tx=0x7f3b08008040 comp rx=0 tx=0).stop 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 msgr2=0x7f3b0c1472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 0x7f3b0c1472f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c4d0 tx=0x7f3afc00c890 comp rx=0 tx=0).stop 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 shutdown_connections 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3b0c15c890 0x7f3b0c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3af00682f0 0x7f3af006a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c06e370 0x7f3b0c1472f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 --2- 172.21.15.83:0/3444801545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b0c0b4bb0 0x7f3b0c0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 >> 172.21.15.83:0/3444801545 conn(0x7f3b0c0b2b60 msgr2=0x7f3b0c0aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 shutdown_connections 2023-11-28T11:10:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.123+0000 7f3b13950700 1 -- 172.21.15.83:0/3444801545 wait complete. 2023-11-28T11:10:23.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.257+0000 7fd57f0a3700 1 Processor -- start 2023-11-28T11:10:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.257+0000 7fd57f0a3700 1 -- start start 2023-11-28T11:10:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.258+0000 7fd57f0a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd578159450 0x7fd57815d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.258+0000 7fd57f0a3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57815deb0 con 0x7fd578070ad0 2023-11-28T11:10:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.258+0000 7fd57f0a3700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57815cf20 con 0x7fd578159f30 2023-11-28T11:10:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.258+0000 7fd57f0a3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57815a480 con 0x7fd578159450 2023-11-28T11:10:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.258+0000 7fd57ce3f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd578159450 0x7fd57815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd577fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd578070ad0 0x7fd578159820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49412/0 (socket says 172.21.15.83:49412) 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd577fff700 1 -- 172.21.15.83:0/458343591 learned_addr learned my addr 172.21.15.83:0/458343591 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd57ce3f700 1 -- 172.21.15.83:0/458343591 >> v1:172.21.15.90:6789/0 conn(0x7fd578159f30 legacy=0x7fd57806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd57ce3f700 1 -- 172.21.15.83:0/458343591 >> v1:172.21.15.83:6789/0 conn(0x7fd578070ad0 legacy=0x7fd578159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd5777fe700 1 -- 172.21.15.83:0/458343591 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939524175 0 0) 0x7fd564001110 con 0x7fd578070ad0 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd57ce3f700 1 -- 172.21.15.83:0/458343591 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57815a760 con 0x7fd578159450 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd57ce3f700 1 --2- 172.21.15.83:0/458343591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159450 0x7fd57815d970 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009870 tx=0x7fd56c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef49e2e8fa7c99f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd5777fe700 1 -- 172.21.15.83:0/458343591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004230 con 0x7fd578159450 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd5777fe700 1 -- 172.21.15.83:0/458343591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c004390 con 0x7fd578159450 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.259+0000 7fd5777fe700 1 -- 172.21.15.83:0/458343591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0036a0 con 0x7fd578159450 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 -- 172.21.15.83:0/458343591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159450 msgr2=0x7fd57815d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/458343591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159450 0x7fd57815d970 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009870 tx=0x7fd56c009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 -- 172.21.15.83:0/458343591 shutdown_connections 2023-11-28T11:10:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/458343591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159450 0x7fd57815d970 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 -- 172.21.15.83:0/458343591 >> 172.21.15.83:0/458343591 conn(0x7fd5780ceb80 msgr2=0x7fd5780cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 -- 172.21.15.83:0/458343591 shutdown_connections 2023-11-28T11:10:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.260+0000 7fd57f0a3700 1 -- 172.21.15.83:0/458343591 wait complete. 2023-11-28T11:10:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.261+0000 7fd57f0a3700 1 Processor -- start 2023-11-28T11:10:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.261+0000 7fd57f0a3700 1 -- start start 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd578070ad0 0x7fd5780c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd578159450 0x7fd5780c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 0x7fd5781550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780c5670 con 0x7fd578159f30 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781f4420 con 0x7fd578070ad0 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781555e0 con 0x7fd578159450 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 0x7fd5781550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 0x7fd5781550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51562/0 (socket says 172.21.15.83:51562) 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 -- 172.21.15.83:0/2667975473 learned_addr learned my addr 172.21.15.83:0/2667975473 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57ce3f700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd578070ad0 0x7fd5780c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd577fff700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd578159450 0x7fd5780c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 -- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd578159450 msgr2=0x7fd5780c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd578159450 0x7fd5780c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 -- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd578070ad0 msgr2=0x7fd5780c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd578070ad0 0x7fd5780c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 -- 172.21.15.83:0/2667975473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0094a0 con 0x7fd578159f30 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd577fff700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd578159450 0x7fd5780c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57d640700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 0x7fd5781550a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b9a0 tx=0x7fd57000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000c9b0 con 0x7fd578159f30 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000cb10 con 0x7fd578159f30 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57815d450 con 0x7fd578159f30 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.262+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570014440 con 0x7fd578159f30 2023-11-28T11:10:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.263+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57815d170 con 0x7fd578159f30 2023-11-28T11:10:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.263+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd578071cb0 con 0x7fd578159f30 2023-11-28T11:10:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.264+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd57001b030 con 0x7fd578159f30 2023-11-28T11:10:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.264+0000 7fd5757fa700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd55c0682f0 0x7fd55c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.264+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd570085670 con 0x7fd578159f30 2023-11-28T11:10:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.264+0000 7fd57ce3f700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd55c0682f0 0x7fd55c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.265+0000 7fd57ce3f700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd55c0682f0 0x7fd55c06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009fd0 tx=0x7fd56c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.269+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd570052060 con 0x7fd578159f30 2023-11-28T11:10:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.450+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5780742b0 con 0x7fd578159f30 2023-11-28T11:10:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.450+0000 7fd5757fa700 1 -- 172.21.15.83:0/2667975473 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd570055680 con 0x7fd578159f30 2023-11-28T11:10:23.452 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:10:23.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd55c0682f0 msgr2=0x7fd55c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd55c0682f0 0x7fd55c06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009fd0 tx=0x7fd56c01c040 comp rx=0 tx=0).stop 2023-11-28T11:10:23.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 msgr2=0x7fd5781550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 0x7fd5781550a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b9a0 tx=0x7fd57000bd60 comp rx=0 tx=0).stop 2023-11-28T11:10:23.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 shutdown_connections 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd55c0682f0 0x7fd55c06a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd578159f30 0x7fd5781550a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd578070ad0 0x7fd5780c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 --2- 172.21.15.83:0/2667975473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd578159450 0x7fd5780c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.452+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 >> 172.21.15.83:0/2667975473 conn(0x7fd5780ceb80 msgr2=0x7fd5780c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.453+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 shutdown_connections 2023-11-28T11:10:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.453+0000 7fd57f0a3700 1 -- 172.21.15.83:0/2667975473 wait complete. 2023-11-28T11:10:23.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:10:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.583+0000 7f657cce6700 1 Processor -- start 2023-11-28T11:10:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.583+0000 7f657cce6700 1 -- start start 2023-11-28T11:10:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.584+0000 7f657cce6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f657815cd10 0x7f65780ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.584+0000 7f657cce6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f65780cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.584+0000 7f657cce6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65780cbfb0 con 0x7f65780b18a0 2023-11-28T11:10:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.584+0000 7f657cce6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657806cee0 con 0x7f657815cd10 2023-11-28T11:10:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.584+0000 7f657cce6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780cb710 con 0x7f657815d0e0 2023-11-28T11:10:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f657815cd10 0x7f65780ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f6576d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f65780b18a0 0x7f65781609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49428/0 (socket says 172.21.15.83:49428) 2023-11-28T11:10:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f6576d9d700 1 -- 172.21.15.83:0/4241304762 learned_addr learned my addr 172.21.15.83:0/4241304762 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f6575d9b700 1 --2- 172.21.15.83:0/4241304762 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f65780cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657659c700 1 -- 172.21.15.83:0/4241304762 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 msgr2=0x7f65780cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657659c700 1 --2- 172.21.15.83:0/4241304762 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f65780cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657659c700 1 -- 172.21.15.83:0/4241304762 >> v1:172.21.15.83:6789/0 conn(0x7f65780b18a0 legacy=0x7f65781609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657659c700 1 -- 172.21.15.83:0/4241304762 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65780cb4e0 con 0x7f657815cd10 2023-11-28T11:10:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657659c700 1 --2- 172.21.15.83:0/4241304762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f65780ca970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f656c009840 tx=0x7f656c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877cc91f84ff9820 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657559a700 1 -- 172.21.15.83:0/4241304762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c011830 con 0x7f657815cd10 2023-11-28T11:10:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657559a700 1 -- 172.21.15.83:0/4241304762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c011990 con 0x7f657815cd10 2023-11-28T11:10:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.585+0000 7f657559a700 1 -- 172.21.15.83:0/4241304762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c011d20 con 0x7f657815cd10 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 -- 172.21.15.83:0/4241304762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 msgr2=0x7f65780ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 --2- 172.21.15.83:0/4241304762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f65780ca970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f656c009840 tx=0x7f656c009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 -- 172.21.15.83:0/4241304762 shutdown_connections 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 --2- 172.21.15.83:0/4241304762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f65780ca970 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 --2- 172.21.15.83:0/4241304762 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f65780cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 -- 172.21.15.83:0/4241304762 >> 172.21.15.83:0/4241304762 conn(0x7f657806e9a0 msgr2=0x7f65780cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 -- 172.21.15.83:0/4241304762 shutdown_connections 2023-11-28T11:10:23.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.586+0000 7f657cce6700 1 -- 172.21.15.83:0/4241304762 wait complete. 2023-11-28T11:10:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.587+0000 7f657cce6700 1 Processor -- start 2023-11-28T11:10:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.587+0000 7f657cce6700 1 -- start start 2023-11-28T11:10:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657cce6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65780b18a0 0x7f65780b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657cce6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f657815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657cce6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f657815fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657cce6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6578160530 con 0x7f657815cd10 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657cce6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65780c8cc0 con 0x7f65780b18a0 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657cce6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780c7c20 con 0x7f657815d0e0 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f657815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f657815fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51574/0 (socket says 172.21.15.83:51574) 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 -- 172.21.15.83:0/3609084692 learned_addr learned my addr 172.21.15.83:0/3609084692 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f657659c700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65780b18a0 0x7f65780b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6576d9d700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f657815fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 -- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 msgr2=0x7f657815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f657815fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 -- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65780b18a0 msgr2=0x7f65780b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65780b18a0 0x7f65780b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 -- 172.21.15.83:0/3609084692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c0094a0 con 0x7f657815cd10 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.588+0000 7f6575d9b700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f657815fab0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f656800c7f0 tx=0x7f656800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.589+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568011800 con 0x7f657815cd10 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.589+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568011960 con 0x7f657815cd10 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.589+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65780cbcd0 con 0x7f657815cd10 2023-11-28T11:10:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.589+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568011cf0 con 0x7f657815cd10 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.589+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65780cb9f0 con 0x7f657815cd10 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.590+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657806b9b0 con 0x7f657815cd10 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.590+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6568021070 con 0x7f657815cd10 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.590+0000 7f6566ffd700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65580682f0 0x7f655806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.590+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f65680858c0 con 0x7f657815cd10 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.591+0000 7f657659c700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65580682f0 0x7f655806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.591+0000 7f657659c700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65580682f0 0x7f655806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f656c0090b0 tx=0x7f656c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.594+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65680522a0 con 0x7f657815cd10 2023-11-28T11:10:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.779+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6578074320 con 0x7f657815cd10 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.779+0000 7f6566ffd700 1 -- 172.21.15.83:0/3609084692 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+126 (secure 0 0 0) 0x7f656801f030 con 0x7f657815cd10 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": "rbd" 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:23.781 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 2, 2023-11-28T11:10:23.782 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-28T11:10:23.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:23.782 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-28T11:10:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65580682f0 msgr2=0x7f655806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65580682f0 0x7f655806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f656c0090b0 tx=0x7f656c008030 comp rx=0 tx=0).stop 2023-11-28T11:10:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 msgr2=0x7f657815fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f657815fab0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f656800c7f0 tx=0x7f656800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 shutdown_connections 2023-11-28T11:10:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f657815d0e0 0x7f657815fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65780b18a0 0x7f65780b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65580682f0 0x7f655806a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 --2- 172.21.15.83:0/3609084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657815cd10 0x7f657815fab0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 >> 172.21.15.83:0/3609084692 conn(0x7f657806e9a0 msgr2=0x7f65780cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 shutdown_connections 2023-11-28T11:10:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.781+0000 7f657cce6700 1 -- 172.21.15.83:0/3609084692 wait complete. 2023-11-28T11:10:23.795 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:10:23.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:10:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.918+0000 7f1edff96700 1 Processor -- start 2023-11-28T11:10:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.918+0000 7f1edff96700 1 -- start start 2023-11-28T11:10:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edff96700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ed806cf30 0x7f1ed80b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edff96700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed815d100 con 0x7f1ed815cd30 2023-11-28T11:10:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edff96700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed806beb0 con 0x7f1ed80cb520 2023-11-28T11:10:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edff96700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed806c170 con 0x7f1ed806cf30 2023-11-28T11:10:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1ede533700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ed806cf30 0x7f1ed80b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1eddd32700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1ed815cd30 0x7f1ed80cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49438/0 (socket says 172.21.15.83:49438) 2023-11-28T11:10:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1eddd32700 1 -- 172.21.15.83:0/4125633306 learned_addr learned my addr 172.21.15.83:0/4125633306 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1ede533700 1 -- 172.21.15.83:0/4125633306 >> v1:172.21.15.90:6789/0 conn(0x7f1ed80cb520 legacy=0x7f1ed80b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1ede533700 1 -- 172.21.15.83:0/4125633306 >> v1:172.21.15.83:6789/0 conn(0x7f1ed815cd30 legacy=0x7f1ed80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1ede533700 1 -- 172.21.15.83:0/4125633306 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed806c450 con 0x7f1ed806cf30 2023-11-28T11:10:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edcd30700 1 -- 172.21.15.83:0/4125633306 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045814561 0 0) 0x7f1ed4001110 con 0x7f1ed815cd30 2023-11-28T11:10:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1ede533700 1 --2- 172.21.15.83:0/4125633306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed806cf30 0x7f1ed80b1ef0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009d90 tx=0x7f1ed000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab22c7c24475e2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edcd30700 1 -- 172.21.15.83:0/4125633306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000e070 con 0x7f1ed806cf30 2023-11-28T11:10:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.919+0000 7f1edcd30700 1 -- 172.21.15.83:0/4125633306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed000bbd0 con 0x7f1ed806cf30 2023-11-28T11:10:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edcd30700 1 -- 172.21.15.83:0/4125633306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0012430 con 0x7f1ed806cf30 2023-11-28T11:10:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 -- 172.21.15.83:0/4125633306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed806cf30 msgr2=0x7f1ed80b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 --2- 172.21.15.83:0/4125633306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed806cf30 0x7f1ed80b1ef0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009d90 tx=0x7f1ed000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 -- 172.21.15.83:0/4125633306 shutdown_connections 2023-11-28T11:10:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 --2- 172.21.15.83:0/4125633306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed806cf30 0x7f1ed80b1ef0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 -- 172.21.15.83:0/4125633306 >> 172.21.15.83:0/4125633306 conn(0x7f1ed80cc690 msgr2=0x7f1ed80cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 -- 172.21.15.83:0/4125633306 shutdown_connections 2023-11-28T11:10:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.920+0000 7f1edff96700 1 -- 172.21.15.83:0/4125633306 wait complete. 2023-11-28T11:10:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.921+0000 7f1edff96700 1 Processor -- start 2023-11-28T11:10:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.921+0000 7f1edff96700 1 -- start start 2023-11-28T11:10:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edff96700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ed806cf30 0x7f1ed80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edff96700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ed80cb520 0x7f1ed80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edff96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 0x7f1ed81f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edff96700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80ad6a0 con 0x7f1ed815cd30 2023-11-28T11:10:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edff96700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80ad980 con 0x7f1ed806cf30 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edff96700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80adc60 con 0x7f1ed80cb520 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 0x7f1ed81f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 0x7f1ed81f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51586/0 (socket says 172.21.15.83:51586) 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 -- 172.21.15.83:0/2516580574 learned_addr learned my addr 172.21.15.83:0/2516580574 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1eddd32700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ed806cf30 0x7f1ed80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1edd531700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ed80cb520 0x7f1ed80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 -- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ed80cb520 msgr2=0x7f1ed80ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ed80cb520 0x7f1ed80ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 -- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ed806cf30 msgr2=0x7f1ed80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ed806cf30 0x7f1ed80ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1ede533700 1 -- 172.21.15.83:0/2516580574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009a70 con 0x7f1ed815cd30 2023-11-28T11:10:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.922+0000 7f1eddd32700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ed806cf30 0x7f1ed80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.923+0000 7f1ede533700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 0x7f1ed81f2430 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0014040 tx=0x7f1ed0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.923+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed001b810 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.923+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed0020070 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.923+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed806ca10 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.923+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed001bbb0 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.923+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed806c730 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.924+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed80742b0 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.924+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ed001bd10 con 0x7f1ed815cd30 2023-11-28T11:10:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.925+0000 7f1ece7fc700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ebc0682f0 0x7f1ebc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.925+0000 7f1eddd32700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ebc0682f0 0x7f1ebc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.925+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1ed0022070 con 0x7f1ed815cd30 2023-11-28T11:10:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.925+0000 7f1eddd32700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ebc0682f0 0x7f1ebc06a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4000f60 tx=0x7f1ed4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:23.929+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ed0055100 con 0x7f1ed815cd30 2023-11-28T11:10:24.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.115+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1ed80b5b90 con 0x7f1ed815cd30 2023-11-28T11:10:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.973+0000 7f1ece7fc700 1 -- 172.21.15.83:0/2516580574 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1ed0058720 con 0x7f1ed815cd30 2023-11-28T11:10:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ebc0682f0 msgr2=0x7f1ebc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ebc0682f0 0x7f1ebc06a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4000f60 tx=0x7f1ed4009040 comp rx=0 tx=0).stop 2023-11-28T11:10:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 msgr2=0x7f1ed81f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 0x7f1ed81f2430 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0014040 tx=0x7f1ed0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 shutdown_connections 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ebc0682f0 0x7f1ebc06a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ed815cd30 0x7f1ed81f2430 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ed806cf30 0x7f1ed80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 --2- 172.21.15.83:0/2516580574 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ed80cb520 0x7f1ed80ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 >> 172.21.15.83:0/2516580574 conn(0x7f1ed80cc690 msgr2=0x7f1ed80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.976+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 shutdown_connections 2023-11-28T11:10:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:24.977+0000 7f1edff96700 1 -- 172.21.15.83:0/2516580574 wait complete. 2023-11-28T11:10:24.980 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-28T11:10:24.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:10:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.127+0000 7fae55832700 1 Processor -- start 2023-11-28T11:10:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.127+0000 7fae55832700 1 -- start start 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae55832700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fae5006adf0 0x7fae5015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae55832700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 0x7fae500b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae55832700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5006a960 con 0x7fae500c64c0 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae55832700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae500b5480 con 0x7fae5006adf0 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae55832700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae500ce130 con 0x7fae500b4ac0 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4effd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fae500c64c0 0x7fae500cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49450/0 (socket says 172.21.15.83:49450) 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4effd700 1 -- 172.21.15.83:0/1831178057 learned_addr learned my addr 172.21.15.83:0/1831178057 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193237483 0 0) 0x7fae5006a960 con 0x7fae500c64c0 2023-11-28T11:10:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae380035b0 con 0x7fae500c64c0 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4e7fc700 1 --2- 172.21.15.83:0/1831178057 >> v2:172.21.15.90:3300/0 conn(0x7fae5006adf0 0x7fae5015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4f7fe700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 0x7fae500b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2609030562 0 0) 0x7fae380035b0 con 0x7fae500c64c0 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 msgr2=0x7fae500b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 0x7fae500b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 >> v2:172.21.15.90:3300/0 conn(0x7fae5006adf0 msgr2=0x7fae5015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 --2- 172.21.15.83:0/1831178057 >> v2:172.21.15.90:3300/0 conn(0x7fae5006adf0 0x7fae5015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500cde50 con 0x7fae500c64c0 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fae44001670 con 0x7fae500c64c0 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.128+0000 7fae4e7fc700 1 --2- 172.21.15.83:0/1831178057 >> v2:172.21.15.90:3300/0 conn(0x7fae5006adf0 0x7fae5015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 >> v1:172.21.15.83:6789/0 conn(0x7fae500c64c0 legacy=0x7fae500cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 0x7fae38003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae38003f80 0x7fae38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae38004d90 0x7fae38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500b5480 con 0x7fae38004d90 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500ce130 con 0x7fae38003f80 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3800baf0 con 0x7fae500b4ac0 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae38004d90 0x7fae38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4effd700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 0x7fae38003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4e7fc700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae38003f80 0x7fae38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 -- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 msgr2=0x7fae38003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae500b4ac0 0x7fae38003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 -- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae38003f80 msgr2=0x7fae38004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae38003f80 0x7fae38004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 -- 172.21.15.83:0/1831178057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae380035b0 con 0x7fae38004d90 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4f7fe700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae38004d90 0x7fae38005200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fae48009f90 tx=0x7fae38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff7dcbe139dba33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4800ec30 con 0x7fae38004d90 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4800f040 con 0x7fae38004d90 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.129+0000 7fae4dffb700 1 -- 172.21.15.83:0/1831178057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48014480 con 0x7fae38004d90 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae55832700 1 -- 172.21.15.83:0/1831178057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae38004d90 msgr2=0x7fae38005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae55832700 1 --2- 172.21.15.83:0/1831178057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae38004d90 0x7fae38005200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fae48009f90 tx=0x7fae38005880 comp rx=0 tx=0).stop 2023-11-28T11:10:25.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae4f7fe700 1 -- 172.21.15.83:0/1831178057 reap_dead start 2023-11-28T11:10:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae55832700 1 -- 172.21.15.83:0/1831178057 shutdown_connections 2023-11-28T11:10:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae55832700 1 -- 172.21.15.83:0/1831178057 >> 172.21.15.83:0/1831178057 conn(0x7fae500cfac0 msgr2=0x7fae500cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae55832700 1 -- 172.21.15.83:0/1831178057 shutdown_connections 2023-11-28T11:10:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.130+0000 7fae55832700 1 -- 172.21.15.83:0/1831178057 wait complete. 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.131+0000 7fae55832700 1 Processor -- start 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.131+0000 7fae55832700 1 -- start start 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae55832700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae501529b0 0x7fae5015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae55832700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 0x7fae500c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae55832700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae50153270 0x7fae50158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae55832700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5015aa60 con 0x7fae50152d80 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae55832700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5015b2f0 con 0x7fae50153270 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae55832700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae500c86d0 con 0x7fae501529b0 2023-11-28T11:10:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 0x7fae500c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 0x7fae500c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51604/0 (socket says 172.21.15.83:51604) 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 -- 172.21.15.83:0/3197912292 learned_addr learned my addr 172.21.15.83:0/3197912292 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4effd700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae501529b0 0x7fae5015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4f7fe700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae50153270 0x7fae50158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 -- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae501529b0 msgr2=0x7fae5015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae501529b0 0x7fae5015a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 -- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae50153270 msgr2=0x7fae50158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae50153270 0x7fae50158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 -- 172.21.15.83:0/3197912292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5006a5f0 con 0x7fae50152d80 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae4e7fc700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 0x7fae500c91b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fae400116b0 tx=0x7fae400119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.132+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40004440 con 0x7fae50152d80 2023-11-28T11:10:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.133+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae400045a0 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.133+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400038a0 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.133+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5006a310 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.133+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae500c6890 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.134+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae500c8500 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.134+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae40025070 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.134+0000 7fae37fff700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae3c0682f0 0x7fae3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.134+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fae40085ed0 con 0x7fae50152d80 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.134+0000 7fae4effd700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae3c0682f0 0x7fae3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.135+0000 7fae4effd700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae3c0682f0 0x7fae3c06a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae440048d0 tx=0x7fae44007980 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.139+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae40052890 con 0x7fae50152d80 2023-11-28T11:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.321+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fae5006b770 con 0x7fae50152d80 2023-11-28T11:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.980+0000 7fae37fff700 1 -- 172.21.15.83:0/3197912292 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7fae40055eb0 con 0x7fae50152d80 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.986+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae3c0682f0 msgr2=0x7fae3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.986+0000 7fae55832700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae3c0682f0 0x7fae3c06a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae440048d0 tx=0x7fae44007980 comp rx=0 tx=0).stop 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.986+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 msgr2=0x7fae500c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.986+0000 7fae55832700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 0x7fae500c91b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fae400116b0 tx=0x7fae400119c0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 shutdown_connections 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae50153270 0x7fae50158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae501529b0 0x7fae5015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae3c0682f0 0x7fae3c06a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 --2- 172.21.15.83:0/3197912292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae50152d80 0x7fae500c91b0 secure :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fae400116b0 tx=0x7fae400119c0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 >> 172.21.15.83:0/3197912292 conn(0x7fae500cfac0 msgr2=0x7fae5015c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.987+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 shutdown_connections 2023-11-28T11:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:25.989+0000 7fae55832700 1 -- 172.21.15.83:0/3197912292 wait complete. 2023-11-28T11:10:25.996 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-28T11:10:26.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:10:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.141+0000 7f660ad74700 1 Processor -- start 2023-11-28T11:10:26.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.141+0000 7f660ad74700 1 -- start start 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f660ad74700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f660415db10 0x7f66040b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f660ad74700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f660415dee0 0x7f66040ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f660ad74700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040ce520 con 0x7f660406d260 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f660ad74700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040b5b00 con 0x7f660415db10 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f660ad74700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040cd910 con 0x7f660415dee0 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6608b10700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f660415db10 0x7f66040b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6609311700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f660406d260 0x7f66040cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49472/0 (socket says 172.21.15.83:49472) 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6609311700 1 -- 172.21.15.83:0/2161244526 learned_addr learned my addr 172.21.15.83:0/2161244526 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6603fff700 1 --2- 172.21.15.83:0/2161244526 >> v2:172.21.15.90:3300/0 conn(0x7f660415dee0 0x7f66040ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6608b10700 1 -- 172.21.15.83:0/2161244526 >> v2:172.21.15.90:3300/0 conn(0x7f660415dee0 msgr2=0x7f66040ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6608b10700 1 --2- 172.21.15.83:0/2161244526 >> v2:172.21.15.90:3300/0 conn(0x7f660415dee0 0x7f66040ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6608b10700 1 -- 172.21.15.83:0/2161244526 >> v1:172.21.15.83:6789/0 conn(0x7f660406d260 legacy=0x7f66040cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f66037fe700 1 -- 172.21.15.83:0/2161244526 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472510433 0 0) 0x7f66040ce520 con 0x7f660406d260 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6608b10700 1 -- 172.21.15.83:0/2161244526 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040cdbf0 con 0x7f660415db10 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6603fff700 1 --2- 172.21.15.83:0/2161244526 >> v2:172.21.15.90:3300/0 conn(0x7f660415dee0 0x7f66040ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f6608b10700 1 --2- 172.21.15.83:0/2161244526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415db10 0x7f66040b55c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f660415f4c0 tx=0x7f65f8009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c82fda5c5d1a41 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.142+0000 7f66037fe700 1 -- 172.21.15.83:0/2161244526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8005e30 con 0x7f660415db10 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f66037fe700 1 -- 172.21.15.83:0/2161244526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8006030 con 0x7f660415db10 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f66037fe700 1 -- 172.21.15.83:0/2161244526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80063c0 con 0x7f660415db10 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 -- 172.21.15.83:0/2161244526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415db10 msgr2=0x7f66040b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 --2- 172.21.15.83:0/2161244526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415db10 0x7f66040b55c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f660415f4c0 tx=0x7f65f8009b00 comp rx=0 tx=0).stop 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 -- 172.21.15.83:0/2161244526 shutdown_connections 2023-11-28T11:10:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 --2- 172.21.15.83:0/2161244526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415db10 0x7f66040b55c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 --2- 172.21.15.83:0/2161244526 >> v2:172.21.15.90:3300/0 conn(0x7f660415dee0 0x7f66040ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 -- 172.21.15.83:0/2161244526 >> 172.21.15.83:0/2161244526 conn(0x7f66040b2b60 msgr2=0x7f66040b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 -- 172.21.15.83:0/2161244526 shutdown_connections 2023-11-28T11:10:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.143+0000 7f660ad74700 1 -- 172.21.15.83:0/2161244526 wait complete. 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.145+0000 7f660ad74700 1 Processor -- start 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.145+0000 7f660ad74700 1 -- start start 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f660406d260 0x7f66040c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f660415db10 0x7f66040cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 0x7f660415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040cd0d0 con 0x7f660415dee0 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c3480 con 0x7f660415db10 2023-11-28T11:10:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c3760 con 0x7f660406d260 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 0x7f660415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 0x7f660415b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51620/0 (socket says 172.21.15.83:51620) 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 -- 172.21.15.83:0/372786098 learned_addr learned my addr 172.21.15.83:0/372786098 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6603fff700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f660415db10 0x7f66040cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6608b10700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f660406d260 0x7f66040c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 -- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f660406d260 msgr2=0x7f66040c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f660406d260 0x7f66040c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 -- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f660415db10 msgr2=0x7f66040cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f660415db10 0x7f66040cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 -- 172.21.15.83:0/372786098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8009430 con 0x7f660415dee0 2023-11-28T11:10:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f6609311700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 0x7f660415b3a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f65fc002230 tx=0x7f65fc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00aac0 con 0x7f660415dee0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00ac40 con 0x7f660415dee0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040ce1b0 con 0x7f660415dee0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.146+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0070d0 con 0x7f660415dee0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.147+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66040cded0 con 0x7f660415dee0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.148+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66040ceef0 con 0x7f660415dee0 2023-11-28T11:10:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.148+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65fc01a030 con 0x7f660415dee0 2023-11-28T11:10:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.148+0000 7f66017fa700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e80682f0 0x7f65e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.148+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f65fc0859d0 con 0x7f660415dee0 2023-11-28T11:10:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.148+0000 7f6608b10700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e80682f0 0x7f65e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.149+0000 7f6608b10700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e80682f0 0x7f65e806a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f65f8000c00 tx=0x7f65f8008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.153+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65fc052150 con 0x7f660415dee0 2023-11-28T11:10:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.380+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f66040b37b0 con 0x7f660415dee0 2023-11-28T11:10:26.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.385+0000 7f66017fa700 1 -- 172.21.15.83:0/372786098 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f65fc0180c0 con 0x7f660415dee0 2023-11-28T11:10:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e80682f0 msgr2=0x7f65e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e80682f0 0x7f65e806a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f65f8000c00 tx=0x7f65f8008070 comp rx=0 tx=0).stop 2023-11-28T11:10:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 msgr2=0x7f660415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 0x7f660415b3a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f65fc002230 tx=0x7f65fc00cf50 comp rx=0 tx=0).stop 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 shutdown_connections 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f65e80682f0 0x7f65e806a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660415dee0 0x7f660415b3a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f660406d260 0x7f66040c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 --2- 172.21.15.83:0/372786098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f660415db10 0x7f66040cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 >> 172.21.15.83:0/372786098 conn(0x7f66040b2b60 msgr2=0x7f66040cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 shutdown_connections 2023-11-28T11:10:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.388+0000 7f660ad74700 1 -- 172.21.15.83:0/372786098 wait complete. 2023-11-28T11:10:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 8 and data pool 9 2023-11-28T11:10:26.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:26.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.534+0000 7f13c7f73700 1 Processor -- start 2023-11-28T11:10:26.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.534+0000 7f13c7f73700 1 -- start start 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c7f73700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13c015c850 0x7f13c00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c7f73700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c7f73700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c0161450 con 0x7f13c015cc20 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c7f73700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c00ce630 con 0x7f13c015c850 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c7f73700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c01608d0 con 0x7f13c006cf90 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c5d0f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13c015c850 0x7f13c00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c5d0f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13c015c850 0x7f13c00ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51624/0 (socket says 172.21.15.83:51624) 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.535+0000 7f13c5d0f700 1 -- 172.21.15.83:0/4241764096 learned_addr learned my addr 172.21.15.83:0/4241764096 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c5d0f700 1 -- 172.21.15.83:0/4241764096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 msgr2=0x7f13c00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c6510700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c5d0f700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c00ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c5d0f700 1 -- 172.21.15.83:0/4241764096 >> v1:172.21.15.90:6789/0 conn(0x7f13c015cc20 legacy=0x7f13c006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c5d0f700 1 -- 172.21.15.83:0/4241764096 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0160bb0 con 0x7f13c015c850 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c6510700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c5d0f700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c00ceab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f13b00098f0 tx=0x7f13b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de38ff45673838d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c4d0d700 1 -- 172.21.15.83:0/4241764096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000b8a0 con 0x7f13c015c850 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c4d0d700 1 -- 172.21.15.83:0/4241764096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000ba00 con 0x7f13c015c850 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.536+0000 7f13c4d0d700 1 -- 172.21.15.83:0/4241764096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000bd90 con 0x7f13c015c850 2023-11-28T11:10:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 -- 172.21.15.83:0/4241764096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 msgr2=0x7f13c00ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c00ceab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f13b00098f0 tx=0x7f13b0009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 -- 172.21.15.83:0/4241764096 shutdown_connections 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 --2- 172.21.15.83:0/4241764096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c00ceab0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 -- 172.21.15.83:0/4241764096 >> 172.21.15.83:0/4241764096 conn(0x7f13c00cd960 msgr2=0x7f13c00cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 -- 172.21.15.83:0/4241764096 shutdown_connections 2023-11-28T11:10:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.537+0000 7f13c7f73700 1 -- 172.21.15.83:0/4241764096 wait complete. 2023-11-28T11:10:26.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.538+0000 7f13c7f73700 1 Processor -- start 2023-11-28T11:10:26.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.538+0000 7f13c7f73700 1 -- start start 2023-11-28T11:10:26.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c01f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13c015cc20 0x7f13c015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00c6280 con 0x7f13c015c850 2023-11-28T11:10:26.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00b6240 con 0x7f13c015cc20 2023-11-28T11:10:26.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00c6ac0 con 0x7f13c006cf90 2023-11-28T11:10:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c015a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51632/0 (socket says 172.21.15.83:51632) 2023-11-28T11:10:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 -- 172.21.15.83:0/1971195553 learned_addr learned my addr 172.21.15.83:0/1971195553 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c5d0f700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c01f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c6510700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13c015cc20 0x7f13c015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 -- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 msgr2=0x7f13c01f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c01f1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 -- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13c015cc20 msgr2=0x7f13c015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13c015cc20 0x7f13c015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 -- 172.21.15.83:0/1971195553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0009550 con 0x7f13c015c850 2023-11-28T11:10:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c550e700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c015a7c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00c450 tx=0x7f13bc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0052d0 con 0x7f13c015c850 2023-11-28T11:10:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc005430 con 0x7f13c015c850 2023-11-28T11:10:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c0161170 con 0x7f13c015c850 2023-11-28T11:10:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0038c0 con 0x7f13c015c850 2023-11-28T11:10:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.539+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c0160e90 con 0x7f13c015c850 2023-11-28T11:10:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.540+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c0160560 con 0x7f13c015c850 2023-11-28T11:10:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.541+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13bc019030 con 0x7f13c015c850 2023-11-28T11:10:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.541+0000 7f13b67fc700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13a40682f0 0x7f13a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.541+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f13bc084f70 con 0x7f13c015c850 2023-11-28T11:10:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.541+0000 7f13c5d0f700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13a40682f0 0x7f13a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.541+0000 7f13c5d0f700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13a40682f0 0x7f13a406a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009c40 tx=0x7f13b0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.545+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13bc051690 con 0x7f13c015c850 2023-11-28T11:10:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.728+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13c0074320 con 0x7f13c015c850 2023-11-28T11:10:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.730+0000 7f13b67fc700 1 -- 172.21.15.83:0/1971195553 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18087 (secure 0 0 0) 0x7f13bc017090 con 0x7f13c015c850 2023-11-28T11:10:26.731 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:26.731 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":50,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:26.381854+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13a40682f0 msgr2=0x7f13a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13a40682f0 0x7f13a406a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009c40 tx=0x7f13b0008030 comp rx=0 tx=0).stop 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 msgr2=0x7f13c015a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c015a7c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00c450 tx=0x7f13bc00c760 comp rx=0 tx=0).stop 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 shutdown_connections 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13c015cc20 0x7f13c015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f13c006cf90 0x7f13c01f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13a40682f0 0x7f13a406a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 --2- 172.21.15.83:0/1971195553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c015c850 0x7f13c015a7c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 >> 172.21.15.83:0/1971195553 conn(0x7f13c00cd960 msgr2=0x7f13c006f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 shutdown_connections 2023-11-28T11:10:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.732+0000 7f13c7f73700 1 -- 172.21.15.83:0/1971195553 wait complete. 2023-11-28T11:10:26.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.864+0000 7fa89d8a5700 1 Processor -- start 2023-11-28T11:10:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.864+0000 7fa89d8a5700 1 -- start start 2023-11-28T11:10:26.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa89d8a5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa898159830 0x7fa8980cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa89d8a5700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa89806c5f0 0x7fa89815de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa89d8a5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8980b18a0 0x7fa8980b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa89d8a5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89815d480 con 0x7fa8980b18a0 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa89d8a5700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980b2260 con 0x7fa89806c5f0 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa89d8a5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89806ca60 con 0x7fa898159830 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8980b18a0 0x7fa8980b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8980b18a0 0x7fa8980b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51642/0 (socket says 172.21.15.83:51642) 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 -- 172.21.15.83:0/2672880554 learned_addr learned my addr 172.21.15.83:0/2672880554 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8967fc700 1 --2- 172.21.15.83:0/2672880554 >> v2:172.21.15.90:3300/0 conn(0x7fa89806c5f0 0x7fa89815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa896ffd700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa898159830 0x7fa8980cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 -- 172.21.15.83:0/2672880554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa898159830 msgr2=0x7fa8980cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa898159830 0x7fa8980cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 -- 172.21.15.83:0/2672880554 >> v2:172.21.15.90:3300/0 conn(0x7fa89806c5f0 msgr2=0x7fa89815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 --2- 172.21.15.83:0/2672880554 >> v2:172.21.15.90:3300/0 conn(0x7fa89806c5f0 0x7fa89815de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 -- 172.21.15.83:0/2672880554 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89815e3d0 con 0x7fa8980b18a0 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa896ffd700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa898159830 0x7fa8980cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.865+0000 7fa8977fe700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa8980b1d20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa8980728d0 tx=0x7fa89000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3e28c72cd96509 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa895ffb700 1 -- 172.21.15.83:0/2672880554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89000e070 con 0x7fa8980b18a0 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa895ffb700 1 -- 172.21.15.83:0/2672880554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89000b910 con 0x7fa8980b18a0 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa895ffb700 1 -- 172.21.15.83:0/2672880554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89000bcc0 con 0x7fa8980b18a0 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 -- 172.21.15.83:0/2672880554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 msgr2=0x7fa8980b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa8980b1d20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa8980728d0 tx=0x7fa89000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 -- 172.21.15.83:0/2672880554 shutdown_connections 2023-11-28T11:10:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa898159830 0x7fa8980cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/2672880554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa8980b1d20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/2672880554 >> v2:172.21.15.90:3300/0 conn(0x7fa89806c5f0 0x7fa89815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 -- 172.21.15.83:0/2672880554 >> 172.21.15.83:0/2672880554 conn(0x7fa89806e9a0 msgr2=0x7fa8980cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 -- 172.21.15.83:0/2672880554 shutdown_connections 2023-11-28T11:10:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.866+0000 7fa89d8a5700 1 -- 172.21.15.83:0/2672880554 wait complete. 2023-11-28T11:10:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.867+0000 7fa89d8a5700 1 Processor -- start 2023-11-28T11:10:26.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 -- start start 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa89806c5f0 0x7fa89814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa898157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa898159830 0x7fa8981583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89814d840 con 0x7fa8980b18a0 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980b5dc0 con 0x7fa898159830 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa89d8a5700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980b4db0 con 0x7fa89806c5f0 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa8967fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa898157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa8967fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa898157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51646/0 (socket says 172.21.15.83:51646) 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.868+0000 7fa8967fc700 1 -- 172.21.15.83:0/3469333715 learned_addr learned my addr 172.21.15.83:0/3469333715 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa896ffd700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa89806c5f0 0x7fa89814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8977fe700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa898159830 0x7fa8981583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8967fc700 1 -- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa89806c5f0 msgr2=0x7fa89814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8967fc700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa89806c5f0 0x7fa89814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8967fc700 1 -- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa898159830 msgr2=0x7fa8981583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8967fc700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa898159830 0x7fa8981583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8967fc700 1 -- 172.21.15.83:0/3469333715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009b30 con 0x7fa8980b18a0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa896ffd700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa89806c5f0 0x7fa89814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa8967fc700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa898157e70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa888010e20 tx=0x7fa88800e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800b930 con 0x7fa8980b18a0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888011650 con 0x7fa8980b18a0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8980cfa70 con 0x7fa8980b18a0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880119e0 con 0x7fa8980b18a0 2023-11-28T11:10:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.869+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980cf790 con 0x7fa8980b18a0 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.870+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89815d690 con 0x7fa8980b18a0 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.870+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa888020030 con 0x7fa8980b18a0 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.871+0000 7fa87ffff700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8840682f0 0x7fa88406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.871+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa8880889a0 con 0x7fa8980b18a0 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.871+0000 7fa896ffd700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8840682f0 0x7fa88406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.871+0000 7fa896ffd700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8840682f0 0x7fa88406a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c960 tx=0x7fa88c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:26.875+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8880550c0 con 0x7fa8980b18a0 2023-11-28T11:10:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.062+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa898074320 con 0x7fa8980b18a0 2023-11-28T11:10:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.063+0000 7fa87ffff700 1 -- 172.21.15.83:0/3469333715 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18087 (secure 0 0 0) 0x7fa88801e090 con 0x7fa8980b18a0 2023-11-28T11:10:27.065 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:27.065 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":50,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:26.381854+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8840682f0 msgr2=0x7fa88406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8840682f0 0x7fa88406a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c960 tx=0x7fa88c008040 comp rx=0 tx=0).stop 2023-11-28T11:10:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 msgr2=0x7fa898157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa898157e70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa888010e20 tx=0x7fa88800e640 comp rx=0 tx=0).stop 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 shutdown_connections 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa898159830 0x7fa8981583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa89806c5f0 0x7fa89814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8840682f0 0x7fa88406a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 --2- 172.21.15.83:0/3469333715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8980b18a0 0x7fa898157e70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.065+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 >> 172.21.15.83:0/3469333715 conn(0x7fa89806e9a0 msgr2=0x7fa898154f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.066+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 shutdown_connections 2023-11-28T11:10:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.066+0000 7fa89d8a5700 1 -- 172.21.15.83:0/3469333715 wait complete. 2023-11-28T11:10:27.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:10:27.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.201+0000 7f49d869e700 1 Processor -- start 2023-11-28T11:10:27.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.201+0000 7f49d869e700 1 -- start start 2023-11-28T11:10:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.201+0000 7f49d869e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f49d006c370 0x7f49d00b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d869e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d00b5b00 con 0x7f49d00af1f0 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d869e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d006c800 con 0x7f49d00aee20 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d869e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d006cae0 con 0x7f49d006c370 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5c39700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f49d006c370 0x7f49d00b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d6c3b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f49d00af1f0 0x7f49d00cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49516/0 (socket says 172.21.15.83:49516) 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d6c3b700 1 -- 172.21.15.83:0/1932026969 learned_addr learned my addr 172.21.15.83:0/1932026969 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5c39700 1 -- 172.21.15.83:0/1932026969 >> v1:172.21.15.90:6789/0 conn(0x7f49d00aee20 legacy=0x7f49d006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5c39700 1 -- 172.21.15.83:0/1932026969 >> v1:172.21.15.83:6789/0 conn(0x7f49d00af1f0 legacy=0x7f49d00cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5c39700 1 -- 172.21.15.83:0/1932026969 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d006cdc0 con 0x7f49d006c370 2023-11-28T11:10:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5438700 1 -- 172.21.15.83:0/1932026969 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103965158 0 0) 0x7f49c8001490 con 0x7f49d00af1f0 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5c39700 1 --2- 172.21.15.83:0/1932026969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d006c370 0x7f49d00b55c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009840 tx=0x7f49cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2cf3b83bb6eee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5438700 1 -- 172.21.15.83:0/1932026969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc004230 con 0x7f49d006c370 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5438700 1 -- 172.21.15.83:0/1932026969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc004390 con 0x7f49d006c370 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.202+0000 7f49d5438700 1 -- 172.21.15.83:0/1932026969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0036a0 con 0x7f49d006c370 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 -- 172.21.15.83:0/1932026969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d006c370 msgr2=0x7f49d00b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 --2- 172.21.15.83:0/1932026969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d006c370 0x7f49d00b55c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009840 tx=0x7f49cc009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 -- 172.21.15.83:0/1932026969 shutdown_connections 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 --2- 172.21.15.83:0/1932026969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d006c370 0x7f49d00b55c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 -- 172.21.15.83:0/1932026969 >> 172.21.15.83:0/1932026969 conn(0x7f49d00b2b60 msgr2=0x7f49d00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 -- 172.21.15.83:0/1932026969 shutdown_connections 2023-11-28T11:10:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.203+0000 7f49d869e700 1 -- 172.21.15.83:0/1932026969 wait complete. 2023-11-28T11:10:27.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.204+0000 7f49d869e700 1 Processor -- start 2023-11-28T11:10:27.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.204+0000 7f49d869e700 1 -- start start 2023-11-28T11:10:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d869e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49d006c370 0x7f49d0152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d869e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49d00aee20 0x7f49d0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d869e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 0x7f49d0152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d869e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d01e6170 con 0x7f49d00af1f0 2023-11-28T11:10:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d869e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015a330 con 0x7f49d006c370 2023-11-28T11:10:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d869e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015a610 con 0x7f49d00aee20 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 0x7f49d0152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 0x7f49d0152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51660/0 (socket says 172.21.15.83:51660) 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 -- 172.21.15.83:0/1240964853 learned_addr learned my addr 172.21.15.83:0/1240964853 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d643a700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49d006c370 0x7f49d0152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d5c39700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49d00aee20 0x7f49d0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 -- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49d00aee20 msgr2=0x7f49d0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49d00aee20 0x7f49d0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 -- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49d006c370 msgr2=0x7f49d0152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49d006c370 0x7f49d0152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 -- 172.21.15.83:0/1240964853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0094a0 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d643a700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49d006c370 0x7f49d0152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d5c39700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49d00aee20 0x7f49d0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49d6c3b700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 0x7f49d0152140 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f49c800cb60 tx=0x7f49c800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.205+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800ab50 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.206+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c800acb0 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.206+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d006fc70 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.206+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8007310 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.206+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d006d0a0 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.207+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d006d380 con 0x7f49d00af1f0 2023-11-28T11:10:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.207+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f49c801a030 con 0x7f49d00af1f0 2023-11-28T11:10:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.207+0000 7f49c6ffd700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49b40682f0 0x7f49b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.207+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f49c80858d0 con 0x7f49d00af1f0 2023-11-28T11:10:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.207+0000 7f49d643a700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49b40682f0 0x7f49b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.208+0000 7f49d643a700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49b40682f0 0x7f49b406a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f49c0007fd0 tx=0x7f49c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.211+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49c8051ff0 con 0x7f49d00af1f0 2023-11-28T11:10:27.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:27.440+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f49d00b5b00 con 0x7f49d00af1f0 2023-11-28T11:10:28.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.393+0000 7f49c6ffd700 1 -- 172.21.15.83:0/1240964853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f49d00b5b00 con 0x7f49d00af1f0 2023-11-28T11:10:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49b40682f0 msgr2=0x7f49b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49b40682f0 0x7f49b406a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f49c0007fd0 tx=0x7f49c0009040 comp rx=0 tx=0).stop 2023-11-28T11:10:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 msgr2=0x7f49d0152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 0x7f49d0152140 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f49c800cb60 tx=0x7f49c800cf20 comp rx=0 tx=0).stop 2023-11-28T11:10:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 shutdown_connections 2023-11-28T11:10:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f49b40682f0 0x7f49b406a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d00af1f0 0x7f49d0152140 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49d006c370 0x7f49d0152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 --2- 172.21.15.83:0/1240964853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49d00aee20 0x7f49d0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 >> 172.21.15.83:0/1240964853 conn(0x7f49d00b2b60 msgr2=0x7f49d006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 shutdown_connections 2023-11-28T11:10:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.396+0000 7f49d869e700 1 -- 172.21.15.83:0/1240964853 wait complete. 2023-11-28T11:10:28.411 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:10:28.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:10:28.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.537+0000 7f8773f53700 1 Processor -- start 2023-11-28T11:10:28.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.538+0000 7f8773f53700 1 -- start start 2023-11-28T11:10:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.538+0000 7f8773f53700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.538+0000 7f8773f53700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f876c0b27e0 0x7f876c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.538+0000 7f8773f53700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0b3660 con 0x7f876c0d0560 2023-11-28T11:10:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.538+0000 7f8773f53700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c0cea60 con 0x7f876c0b27e0 2023-11-28T11:10:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.538+0000 7f8773f53700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c15ba90 con 0x7f876c158040 2023-11-28T11:10:28.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87724f0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f876c0b27e0 0x7f876c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:28.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87714ee700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f876c0d0560 0x7f876c0ccfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49526/0 (socket says 172.21.15.83:49526) 2023-11-28T11:10:28.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87714ee700 1 -- 172.21.15.83:0/1888445919 learned_addr learned my addr 172.21.15.83:0/1888445919 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87724f0700 1 -- 172.21.15.83:0/1888445919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 msgr2=0x7f876c15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87724f0700 1 --2- 172.21.15.83:0/1888445919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87724f0700 1 -- 172.21.15.83:0/1888445919 >> v1:172.21.15.83:6789/0 conn(0x7f876c0d0560 legacy=0x7f876c0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87724f0700 1 -- 172.21.15.83:0/1888445919 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0b3180 con 0x7f876c0b27e0 2023-11-28T11:10:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f8770ced700 1 -- 172.21.15.83:0/1888445919 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739450744 0 0) 0x7f876c0b3660 con 0x7f876c0d0560 2023-11-28T11:10:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f87724f0700 1 --2- 172.21.15.83:0/1888445919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0b27e0 0x7f876c0b2c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8764009d90 tx=0x7f876400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22869d48dd66372b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f8770ced700 1 -- 172.21.15.83:0/1888445919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400e070 con 0x7f876c0b27e0 2023-11-28T11:10:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f8770ced700 1 -- 172.21.15.83:0/1888445919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876400bbd0 con 0x7f876c0b27e0 2023-11-28T11:10:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.539+0000 7f8770ced700 1 -- 172.21.15.83:0/1888445919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764012430 con 0x7f876c0b27e0 2023-11-28T11:10:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 -- 172.21.15.83:0/1888445919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0b27e0 msgr2=0x7f876c0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 --2- 172.21.15.83:0/1888445919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0b27e0 0x7f876c0b2c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8764009d90 tx=0x7f876400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 -- 172.21.15.83:0/1888445919 shutdown_connections 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 --2- 172.21.15.83:0/1888445919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0b27e0 0x7f876c0b2c40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 --2- 172.21.15.83:0/1888445919 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 -- 172.21.15.83:0/1888445919 >> 172.21.15.83:0/1888445919 conn(0x7f876c06de20 msgr2=0x7f876c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 -- 172.21.15.83:0/1888445919 shutdown_connections 2023-11-28T11:10:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.540+0000 7f8773f53700 1 -- 172.21.15.83:0/1888445919 wait complete. 2023-11-28T11:10:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.541+0000 7f8773f53700 1 Processor -- start 2023-11-28T11:10:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.541+0000 7f8773f53700 1 -- start start 2023-11-28T11:10:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f876c0b27e0 0x7f876c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 0x7f876c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1e5d20 con 0x7f876c0d0560 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1e6000 con 0x7f876c0b27e0 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0c48f0 con 0x7f876c158040 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 0x7f876c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 0x7f876c158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51672/0 (socket says 172.21.15.83:51672) 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 -- 172.21.15.83:0/1063897995 learned_addr learned my addr 172.21.15.83:0/1063897995 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87724f0700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8771cef700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f876c0b27e0 0x7f876c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 -- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 msgr2=0x7f876c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 -- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f876c0b27e0 msgr2=0x7f876c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f876c0b27e0 0x7f876c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 -- 172.21.15.83:0/1063897995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764009a70 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87724f0700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87714ee700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 0x7f876c158410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f876800c540 tx=0x7f876800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768004c80 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768004de0 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c0cb450 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.542+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768007310 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.543+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c0d0a00 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.544+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c0cb730 con 0x7f876c0d0560 2023-11-28T11:10:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.544+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f876801a020 con 0x7f876c0d0560 2023-11-28T11:10:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.544+0000 7f87627fc700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87500682f0 0x7f875006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.544+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8768085980 con 0x7f876c0d0560 2023-11-28T11:10:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.544+0000 7f8771cef700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87500682f0 0x7f875006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.545+0000 7f8771cef700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87500682f0 0x7f875006a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f875c009a10 tx=0x7f875c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.548+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8768052150 con 0x7f876c0d0560 2023-11-28T11:10:28.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:28.775+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f876c0c4e10 con 0x7f876c0d0560 2023-11-28T11:10:29.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.397+0000 7f87627fc700 1 -- 172.21.15.83:0/1063897995 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f876c0c4e10 con 0x7f876c0d0560 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87500682f0 msgr2=0x7f875006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87500682f0 0x7f875006a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f875c009a10 tx=0x7f875c009190 comp rx=0 tx=0).stop 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 msgr2=0x7f876c158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 0x7f876c158410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f876800c540 tx=0x7f876800c850 comp rx=0 tx=0).stop 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 shutdown_connections 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f876c158040 0x7f876c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f876c0b27e0 0x7f876c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87500682f0 0x7f875006a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 --2- 172.21.15.83:0/1063897995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f876c0d0560 0x7f876c158410 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 >> 172.21.15.83:0/1063897995 conn(0x7f876c06de20 msgr2=0x7f876c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 shutdown_connections 2023-11-28T11:10:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.400+0000 7f8773f53700 1 -- 172.21.15.83:0/1063897995 wait complete. 2023-11-28T11:10:29.415 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:10:29.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.540+0000 7f811544b700 1 Processor -- start 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.540+0000 7f811544b700 1 -- start start 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f811544b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f811006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f811544b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f81100ccde0 0x7f81100cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f811544b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100af610 con 0x7f811006d170 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f811544b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811006c3b0 con 0x7f81100ccde0 2023-11-28T11:10:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f811544b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811006c840 con 0x7f811015cde0 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810e7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f811006d170 0x7f81100cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49540/0 (socket says 172.21.15.83:49540) 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810e7fc700 1 -- 172.21.15.83:0/2515532453 learned_addr learned my addr 172.21.15.83:0/2515532453 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810effd700 1 --2- 172.21.15.83:0/2515532453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f811006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810f7fe700 1 --2- 172.21.15.83:0/2515532453 >> v2:172.21.15.90:3300/0 conn(0x7f81100ccde0 0x7f81100cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810dffb700 1 -- 172.21.15.83:0/2515532453 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193891016 0 0) 0x7f81100af610 con 0x7f811006d170 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810dffb700 1 -- 172.21.15.83:0/2515532453 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f80035b0 con 0x7f811006d170 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810dffb700 1 -- 172.21.15.83:0/2515532453 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003164081 0 0) 0x7f80f80035b0 con 0x7f811006d170 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810effd700 1 -- 172.21.15.83:0/2515532453 >> v2:172.21.15.90:3300/0 conn(0x7f81100ccde0 msgr2=0x7f81100cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810effd700 1 --2- 172.21.15.83:0/2515532453 >> v2:172.21.15.90:3300/0 conn(0x7f81100ccde0 0x7f81100cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810effd700 1 -- 172.21.15.83:0/2515532453 >> v1:172.21.15.83:6789/0 conn(0x7f811006d170 legacy=0x7f81100cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810effd700 1 -- 172.21.15.83:0/2515532453 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811006b7a0 con 0x7f811015cde0 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.541+0000 7f810f7fe700 1 --2- 172.21.15.83:0/2515532453 >> v2:172.21.15.90:3300/0 conn(0x7f81100ccde0 0x7f81100cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f810effd700 1 --2- 172.21.15.83:0/2515532453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f811006cc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8104009840 tx=0x7f8104009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6ac39df8fa2a50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f810dffb700 1 -- 172.21.15.83:0/2515532453 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100001670 con 0x7f811015cde0 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f810dffb700 1 -- 172.21.15.83:0/2515532453 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8104004030 con 0x7f811015cde0 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f810dffb700 1 -- 172.21.15.83:0/2515532453 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81040043c0 con 0x7f811015cde0 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 -- 172.21.15.83:0/2515532453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 msgr2=0x7f811006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 --2- 172.21.15.83:0/2515532453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f811006cc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8104009840 tx=0x7f8104009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 -- 172.21.15.83:0/2515532453 shutdown_connections 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 --2- 172.21.15.83:0/2515532453 >> v2:172.21.15.90:3300/0 conn(0x7f81100ccde0 0x7f81100cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 --2- 172.21.15.83:0/2515532453 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f811006cc30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 -- 172.21.15.83:0/2515532453 >> 172.21.15.83:0/2515532453 conn(0x7f81100b1830 msgr2=0x7f81100b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:29.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 -- 172.21.15.83:0/2515532453 shutdown_connections 2023-11-28T11:10:29.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.542+0000 7f811544b700 1 -- 172.21.15.83:0/2515532453 wait complete. 2023-11-28T11:10:29.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.544+0000 7f811544b700 1 Processor -- start 2023-11-28T11:10:29.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.544+0000 7f811544b700 1 -- start start 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.544+0000 7f811544b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 0x7f81101f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.544+0000 7f811544b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81100ccde0 0x7f811015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.544+0000 7f811544b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f81101f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.544+0000 7f811544b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8110150a10 con 0x7f811006d170 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f811544b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100c37f0 con 0x7f81100ccde0 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f811544b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81101f2570 con 0x7f811015cde0 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 0x7f81101f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 0x7f81101f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51684/0 (socket says 172.21.15.83:51684) 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 -- 172.21.15.83:0/2855012369 learned_addr learned my addr 172.21.15.83:0/2855012369 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810f7fe700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f81101f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810e7fc700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81100ccde0 0x7f811015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 -- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 msgr2=0x7f81101f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f81101f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 -- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81100ccde0 msgr2=0x7f811015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81100ccde0 0x7f811015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 -- 172.21.15.83:0/2855012369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108009a70 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f810effd700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 0x7f81101f2030 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f810400a010 tx=0x7f8104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104003c00 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810401f070 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81040094a0 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104011630 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.545+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811006bd60 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.546+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811006ba80 con 0x7f811006d170 2023-11-28T11:10:29.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.547+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f810402a020 con 0x7f811006d170 2023-11-28T11:10:29.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.547+0000 7f80f77fe700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f80fc0682f0 0x7f80fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.547+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f81040872b0 con 0x7f811006d170 2023-11-28T11:10:29.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.547+0000 7f810e7fc700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f80fc0682f0 0x7f80fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.547+0000 7f810e7fc700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f80fc0682f0 0x7f80fc06a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81000011e0 tx=0x7f8100009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:29.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.551+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81040539d0 con 0x7f811006d170 2023-11-28T11:10:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.774+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f811006c040 con 0x7f811006d170 2023-11-28T11:10:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.774+0000 7f80f77fe700 1 -- 172.21.15.83:0/2855012369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f8104056ff0 con 0x7f811006d170 2023-11-28T11:10:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:29.776 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f80fc0682f0 msgr2=0x7f80fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f80fc0682f0 0x7f80fc06a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81000011e0 tx=0x7f8100009040 comp rx=0 tx=0).stop 2023-11-28T11:10:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 msgr2=0x7f81101f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 0x7f81101f2030 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f810400a010 tx=0x7f8104004750 comp rx=0 tx=0).stop 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 shutdown_connections 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f811015cde0 0x7f81101f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f80fc0682f0 0x7f80fc06a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f811006d170 0x7f81101f2030 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 --2- 172.21.15.83:0/2855012369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81100ccde0 0x7f811015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 >> 172.21.15.83:0/2855012369 conn(0x7f81100b1830 msgr2=0x7f81100cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 shutdown_connections 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.776+0000 7f811544b700 1 -- 172.21.15.83:0/2855012369 wait complete. 2023-11-28T11:10:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:10:29.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:29.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.908+0000 7fefa1b04700 1 Processor -- start 2023-11-28T11:10:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.908+0000 7fefa1b04700 1 -- start start 2023-11-28T11:10:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fefa1b04700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fefa1b04700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef9c06ab00 0x7fef9c06af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fefa1b04700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c06c440 con 0x7fef9c06b8b0 2023-11-28T11:10:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fefa1b04700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c06b4b0 con 0x7fef9c06ab00 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fefa1b04700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0cb220 con 0x7fef9c15aad0 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef9c06ab00 0x7fef9c06af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fef9c06b8b0 0x7fef9c06bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49556/0 (socket says 172.21.15.83:49556) 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9bfff700 1 -- 172.21.15.83:0/3699526279 learned_addr learned my addr 172.21.15.83:0/3699526279 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9b7fe700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9affd700 1 -- 172.21.15.83:0/3699526279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 msgr2=0x7fef9c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9affd700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9affd700 1 -- 172.21.15.83:0/3699526279 >> v1:172.21.15.83:6789/0 conn(0x7fef9c06b8b0 legacy=0x7fef9c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9affd700 1 -- 172.21.15.83:0/3699526279 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c0b33d0 con 0x7fef9c06ab00 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9a7fc700 1 -- 172.21.15.83:0/3699526279 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307638446 0 0) 0x7fef9c06c440 con 0x7fef9c06b8b0 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.909+0000 7fef9b7fe700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fef9affd700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c06af70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e6cdba91ff430b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fef9a7fc700 1 -- 172.21.15.83:0/3699526279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011830 con 0x7fef9c06ab00 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fef9a7fc700 1 -- 172.21.15.83:0/3699526279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c011990 con 0x7fef9c06ab00 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fef9a7fc700 1 -- 172.21.15.83:0/3699526279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011d20 con 0x7fef9c06ab00 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 -- 172.21.15.83:0/3699526279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 msgr2=0x7fef9c06af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c06af70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 -- 172.21.15.83:0/3699526279 shutdown_connections 2023-11-28T11:10:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3699526279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c06af70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 -- 172.21.15.83:0/3699526279 >> 172.21.15.83:0/3699526279 conn(0x7fef9c06e9a0 msgr2=0x7fef9c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 -- 172.21.15.83:0/3699526279 shutdown_connections 2023-11-28T11:10:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.910+0000 7fefa1b04700 1 -- 172.21.15.83:0/3699526279 wait complete. 2023-11-28T11:10:29.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.912+0000 7fefa1b04700 1 Processor -- start 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.912+0000 7fefa1b04700 1 -- start start 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fefa1b04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fefa1b04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c06b8b0 0x7fef9c1f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fefa1b04700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fefa1b04700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f14e0 con 0x7fef9c06ab00 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fefa1b04700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0c43b0 con 0x7fef9c06b8b0 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fefa1b04700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1528b0 con 0x7fef9c15aad0 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51698/0 (socket says 172.21.15.83:51698) 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 -- 172.21.15.83:0/3701295730 learned_addr learned my addr 172.21.15.83:0/3701295730 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9affd700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c06b8b0 0x7fef9c1f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9bfff700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 -- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 msgr2=0x7fef9c0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 -- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c06b8b0 msgr2=0x7fef9c1f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:29.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c06b8b0 0x7fef9c1f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 -- 172.21.15.83:0/3701295730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094a0 con 0x7fef9c06ab00 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9affd700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c06b8b0 0x7fef9c1f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef9b7fe700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c152370 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e840 tx=0x7fef9000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.913+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004230 con 0x7fef9c06ab00 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.914+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90004390 con 0x7fef9c06ab00 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.914+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c0ca610 con 0x7fef9c06ab00 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.914+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900036a0 con 0x7fef9c06ab00 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.914+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c15aea0 con 0x7fef9c06ab00 2023-11-28T11:10:29.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.914+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0ca8f0 con 0x7fef9c06ab00 2023-11-28T11:10:29.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.915+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef90022070 con 0x7fef9c06ab00 2023-11-28T11:10:29.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.916+0000 7fef83fff700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef880682f0 0x7fef8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:29.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.916+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fef90085c80 con 0x7fef9c06ab00 2023-11-28T11:10:29.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.916+0000 7fef9affd700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef880682f0 0x7fef8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:29.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.916+0000 7fef9affd700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef880682f0 0x7fef8806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0090b0 tx=0x7fef8c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:29.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:29.919+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef90055e70 con 0x7fef9c06ab00 2023-11-28T11:10:30.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.103+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef9c074320 con 0x7fef9c06ab00 2023-11-28T11:10:30.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.105+0000 7fef83fff700 1 -- 172.21.15.83:0/3701295730 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18074 (secure 0 0 0) 0x7fef90020090 con 0x7fef9c06ab00 2023-11-28T11:10:30.106 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:30.106 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:27.384310+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef880682f0 msgr2=0x7fef8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef880682f0 0x7fef8806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0090b0 tx=0x7fef8c01c040 comp rx=0 tx=0).stop 2023-11-28T11:10:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 msgr2=0x7fef9c152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c152370 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e840 tx=0x7fef9000ec00 comp rx=0 tx=0).stop 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 shutdown_connections 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef9c15aad0 0x7fef9c0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef880682f0 0x7fef8806a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c06ab00 0x7fef9c152370 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 --2- 172.21.15.83:0/3701295730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c06b8b0 0x7fef9c1f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 >> 172.21.15.83:0/3701295730 conn(0x7fef9c06e9a0 msgr2=0x7fef9c0b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 shutdown_connections 2023-11-28T11:10:30.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.107+0000 7fefa1b04700 1 -- 172.21.15.83:0/3701295730 wait complete. 2023-11-28T11:10:30.120 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:10:30.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.239+0000 7f01affc3700 1 Processor -- start 2023-11-28T11:10:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.239+0000 7f01affc3700 1 -- start start 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01affc3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f01a80b1e30 0x7f01a8167670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01affc3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80ce620 con 0x7f01a8161020 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01affc3700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80ce8c0 con 0x7f01a80af030 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01affc3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80cb820 con 0x7f01a80b1e30 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01ad55e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f01a8161020 0x7f01a80b1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49566/0 (socket says 172.21.15.83:49566) 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01ad55e700 1 -- 172.21.15.83:0/1553548542 learned_addr learned my addr 172.21.15.83:0/1553548542 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01ae560700 1 --2- 172.21.15.83:0/1553548542 >> v2:172.21.15.90:3300/0 conn(0x7f01a80b1e30 0x7f01a8167670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577800978 0 0) 0x7f01a80ce620 con 0x7f01a8161020 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01940035b0 con 0x7f01a8161020 2023-11-28T11:10:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188699556 0 0) 0x7f01940035b0 con 0x7f01a8161020 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 >> v2:172.21.15.90:3300/0 conn(0x7f01a80b1e30 msgr2=0x7f01a8167670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 --2- 172.21.15.83:0/1553548542 >> v2:172.21.15.90:3300/0 conn(0x7f01a80b1e30 0x7f01a8167670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 >> v1:172.21.15.90:6789/0 conn(0x7f01a80af030 legacy=0x7f01a80cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80ceba0 con 0x7f01a8161020 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f01a4001670 con 0x7f01a8161020 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 >> v1:172.21.15.83:6789/0 conn(0x7f01a8161020 legacy=0x7f01a80b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f0194003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.240+0000 7f01acd5d700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0194004850 0x7f0194004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0194005200 0x7f019400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80cb820 con 0x7f0194003a00 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019400c080 con 0x7f0194005200 2023-11-28T11:10:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019400c240 con 0x7f0194004850 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f0194003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01ae560700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0194005200 0x7f019400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01ad55e700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0194004850 0x7f0194004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 -- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0194004850 msgr2=0x7f0194004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0194004850 0x7f0194004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 -- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0194005200 msgr2=0x7f019400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0194005200 0x7f019400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 -- 172.21.15.83:0/1553548542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940035b0 con 0x7f0194003a00 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01ae560700 1 -- 172.21.15.83:0/1553548542 reap_dead start 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01ad55e700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0194004850 0x7f0194004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01add5f700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f0194003e10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f01980017c0 tx=0x7f0194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5103c8a45af7884 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980052d0 con 0x7f0194003a00 2023-11-28T11:10:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0198005450 con 0x7f0194003a00 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.241+0000 7f01acd5d700 1 -- 172.21.15.83:0/1553548542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980038c0 con 0x7f0194003a00 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 -- 172.21.15.83:0/1553548542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 msgr2=0x7f0194003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f0194003e10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f01980017c0 tx=0x7f0194005880 comp rx=0 tx=0).stop 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 -- 172.21.15.83:0/1553548542 shutdown_connections 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 --2- 172.21.15.83:0/1553548542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f0194003e10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 -- 172.21.15.83:0/1553548542 >> 172.21.15.83:0/1553548542 conn(0x7f01a80cc720 msgr2=0x7f01a80ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 -- 172.21.15.83:0/1553548542 shutdown_connections 2023-11-28T11:10:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.242+0000 7f01affc3700 1 -- 172.21.15.83:0/1553548542 wait complete. 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.243+0000 7f01affc3700 1 Processor -- start 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.243+0000 7f01affc3700 1 -- start start 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f01a80b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a80ca4e0 0x7f01a80b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a80acaf0 0x7f01a80acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80aba50 con 0x7f0194003a00 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80abd30 con 0x7f01a80acaf0 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80ac010 con 0x7f01a80ca4e0 2023-11-28T11:10:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f01a80b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f01a80b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51716/0 (socket says 172.21.15.83:51716) 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 -- 172.21.15.83:0/2244248409 learned_addr learned my addr 172.21.15.83:0/2244248409 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01ae560700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a80acaf0 0x7f01a80acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01ad55e700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a80ca4e0 0x7f01a80b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 -- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a80ca4e0 msgr2=0x7f01a80b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a80ca4e0 0x7f01a80b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 -- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a80acaf0 msgr2=0x7f01a80acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a80acaf0 0x7f01a80acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 -- 172.21.15.83:0/2244248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80cf160 con 0x7f0194003a00 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01ae560700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a80acaf0 0x7f01a80acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01add5f700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f01a80b34b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0198000c00 tx=0x7f0198001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198001290 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0198007040 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a80cee80 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980073f0 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.244+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80af400 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.245+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a80ac940 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.246+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f019802d070 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.246+0000 7f019e7fc700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f018c0682f0 0x7f018c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.246+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0198086650 con 0x7f0194003a00 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.246+0000 7f01ad55e700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f018c0682f0 0x7f018c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.246+0000 7f01ad55e700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f018c0682f0 0x7f018c06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f01a80b3f30 tx=0x7f01a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.250+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0198052d70 con 0x7f0194003a00 2023-11-28T11:10:30.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.428+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01a8074320 con 0x7f018c0682f0 2023-11-28T11:10:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.437+0000 7f019e7fc700 1 -- 172.21.15.83:0/2244248409 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311366 (secure 0 0 0) 0x7f01a8074320 con 0x7f018c0682f0 2023-11-28T11:10:30.442 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f018c0682f0 msgr2=0x7f018c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f018c0682f0 0x7f018c06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f01a80b3f30 tx=0x7f01a4009040 comp rx=0 tx=0).stop 2023-11-28T11:10:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 msgr2=0x7f01a80b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f01a80b34b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0198000c00 tx=0x7f0198001120 comp rx=0 tx=0).stop 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 shutdown_connections 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a80acaf0 0x7f01a80acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f018c0682f0 0x7f018c06a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0194003a00 0x7f01a80b34b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 --2- 172.21.15.83:0/2244248409 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a80ca4e0 0x7f01a80b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 >> 172.21.15.83:0/2244248409 conn(0x7f01a80cc720 msgr2=0x7f01a80b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 shutdown_connections 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.440+0000 7f01affc3700 1 -- 172.21.15.83:0/2244248409 wait complete. 2023-11-28T11:10:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:10:30.453 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-11-28T11:10:29.592276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":75432,"kb_used_data":5928,"kb_used_omap":0,"kb_used_meta":69440,"kb_avail":754899288,"statfs":{"total":773094113280,"available":773016870912,"internally_reserved":0,"allocated":6070272,"data_stored":3410232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71106560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.620219"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.391193+0000","last_change":"2023-11-28T11:10:26.414517+0000","last_active":"2023-11-28T11:10:27.391193+0000","last_peered":"2023-11-28T11:10:27.391193+0000","last_clean":"2023-11-28T11:10:27.391193+0000","last_became_active":"2023-11-28T11:10:26.414258+0000","last_became_peered":"2023-11-28T11:10:26.414258+0000","last_unstale":"2023-11-28T11:10:27.391193+0000","last_undegraded":"2023-11-28T11:10:27.391193+0000","last_fullsized":"2023-11-28T11:10:27.391193+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:21.514243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392499+0000","last_change":"2023-11-28T11:10:26.414992+0000","last_active":"2023-11-28T11:10:27.392499+0000","last_peered":"2023-11-28T11:10:27.392499+0000","last_clean":"2023-11-28T11:10:27.392499+0000","last_became_active":"2023-11-28T11:10:26.413263+0000","last_became_peered":"2023-11-28T11:10:26.413263+0000","last_unstale":"2023-11-28T11:10:27.392499+0000","last_undegraded":"2023-11-28T11:10:27.392499+0000","last_fullsized":"2023-11-28T11:10:27.392499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:11.158839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281564+0000","last_change":"2023-11-28T11:10:26.404834+0000","last_active":"2023-11-28T11:10:28.281564+0000","last_peered":"2023-11-28T11:10:28.281564+0000","last_clean":"2023-11-28T11:10:28.281564+0000","last_became_active":"2023-11-28T11:10:26.404563+0000","last_became_peered":"2023-11-28T11:10:26.404563+0000","last_unstale":"2023-11-28T11:10:28.281564+0000","last_undegraded":"2023-11-28T11:10:28.281564+0000","last_fullsized":"2023-11-28T11:10:28.281564+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:26.886535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392916+0000","last_change":"2023-11-28T11:10:26.405022+0000","last_active":"2023-11-28T11:10:27.392916+0000","last_peered":"2023-11-28T11:10:27.392916+0000","last_clean":"2023-11-28T11:10:27.392916+0000","last_became_active":"2023-11-28T11:10:26.402208+0000","last_became_peered":"2023-11-28T11:10:26.402208+0000","last_unstale":"2023-11-28T11:10:27.392916+0000","last_undegraded":"2023-11-28T11:10:27.392916+0000","last_fullsized":"2023-11-28T11:10:27.392916+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:26.886535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.391215+0000","last_change":"2023-11-28T11:10:26.395758+0000","last_active":"2023-11-28T11:10:27.391215+0000","last_peered":"2023-11-28T11:10:27.391215+0000","last_clean":"2023-11-28T11:10:27.391215+0000","last_became_active":"2023-11-28T11:10:26.395449+0000","last_became_peered":"2023-11-28T11:10:26.395449+0000","last_unstale":"2023-11-28T11:10:27.391215+0000","last_undegraded":"2023-11-28T11:10:27.391215+0000","last_fullsized":"2023-11-28T11:10:27.391215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:17.210757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280468+0000","last_change":"2023-11-28T11:10:26.397024+0000","last_active":"2023-11-28T11:10:28.280468+0000","last_peered":"2023-11-28T11:10:28.280468+0000","last_clean":"2023-11-28T11:10:28.280468+0000","last_became_active":"2023-11-28T11:10:26.396792+0000","last_became_peered":"2023-11-28T11:10:26.396792+0000","last_unstale":"2023-11-28T11:10:28.280468+0000","last_undegraded":"2023-11-28T11:10:28.280468+0000","last_fullsized":"2023-11-28T11:10:28.280468+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:27.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281161+0000","last_change":"2023-11-28T11:10:26.413044+0000","last_active":"2023-11-28T11:10:28.281161+0000","last_peered":"2023-11-28T11:10:28.281161+0000","last_clean":"2023-11-28T11:10:28.281161+0000","last_became_active":"2023-11-28T11:10:26.412783+0000","last_became_peered":"2023-11-28T11:10:26.412783+0000","last_unstale":"2023-11-28T11:10:28.281161+0000","last_undegraded":"2023-11-28T11:10:28.281161+0000","last_fullsized":"2023-11-28T11:10:28.281161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:19.705887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392284+0000","last_change":"2023-11-28T11:10:26.409134+0000","last_active":"2023-11-28T11:10:27.392284+0000","last_peered":"2023-11-28T11:10:27.392284+0000","last_clean":"2023-11-28T11:10:27.392284+0000","last_became_active":"2023-11-28T11:10:26.408858+0000","last_became_peered":"2023-11-28T11:10:26.408858+0000","last_unstale":"2023-11-28T11:10:27.392284+0000","last_undegraded":"2023-11-28T11:10:27.392284+0000","last_fullsized":"2023-11-28T11:10:27.392284+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:17.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282084+0000","last_change":"2023-11-28T11:10:26.401800+0000","last_active":"2023-11-28T11:10:28.282084+0000","last_peered":"2023-11-28T11:10:28.282084+0000","last_clean":"2023-11-28T11:10:28.282084+0000","last_became_active":"2023-11-28T11:10:26.401523+0000","last_became_peered":"2023-11-28T11:10:26.401523+0000","last_unstale":"2023-11-28T11:10:28.282084+0000","last_undegraded":"2023-11-28T11:10:28.282084+0000","last_fullsized":"2023-11-28T11:10:28.282084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:46.198121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491525+0000","last_change":"2023-11-28T11:10:26.406955+0000","last_active":"2023-11-28T11:10:27.491525+0000","last_peered":"2023-11-28T11:10:27.491525+0000","last_clean":"2023-11-28T11:10:27.491525+0000","last_became_active":"2023-11-28T11:10:26.406556+0000","last_became_peered":"2023-11-28T11:10:26.406556+0000","last_unstale":"2023-11-28T11:10:27.491525+0000","last_undegraded":"2023-11-28T11:10:27.491525+0000","last_fullsized":"2023-11-28T11:10:27.491525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:07.639448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491523+0000","last_change":"2023-11-28T11:10:26.410342+0000","last_active":"2023-11-28T11:10:27.491523+0000","last_peered":"2023-11-28T11:10:27.491523+0000","last_clean":"2023-11-28T11:10:27.491523+0000","last_became_active":"2023-11-28T11:10:26.410076+0000","last_became_peered":"2023-11-28T11:10:26.410076+0000","last_unstale":"2023-11-28T11:10:27.491523+0000","last_undegraded":"2023-11-28T11:10:27.491523+0000","last_fullsized":"2023-11-28T11:10:27.491523+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:22.685597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393090+0000","last_change":"2023-11-28T11:10:26.405323+0000","last_active":"2023-11-28T11:10:27.393090+0000","last_peered":"2023-11-28T11:10:27.393090+0000","last_clean":"2023-11-28T11:10:27.393090+0000","last_became_active":"2023-11-28T11:10:26.404636+0000","last_became_peered":"2023-11-28T11:10:26.404636+0000","last_unstale":"2023-11-28T11:10:27.393090+0000","last_undegraded":"2023-11-28T11:10:27.393090+0000","last_fullsized":"2023-11-28T11:10:27.393090+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:54.076510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282359+0000","last_change":"2023-11-28T11:10:26.407794+0000","last_active":"2023-11-28T11:10:28.282359+0000","last_peered":"2023-11-28T11:10:28.282359+0000","last_clean":"2023-11-28T11:10:28.282359+0000","last_became_active":"2023-11-28T11:10:26.407527+0000","last_became_peered":"2023-11-28T11:10:26.407527+0000","last_unstale":"2023-11-28T11:10:28.282359+0000","last_undegraded":"2023-11-28T11:10:28.282359+0000","last_fullsized":"2023-11-28T11:10:28.282359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:39.295444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281568+0000","last_change":"2023-11-28T11:10:26.404941+0000","last_active":"2023-11-28T11:10:28.281568+0000","last_peered":"2023-11-28T11:10:28.281568+0000","last_clean":"2023-11-28T11:10:28.281568+0000","last_became_active":"2023-11-28T11:10:26.404641+0000","last_became_peered":"2023-11-28T11:10:26.404641+0000","last_unstale":"2023-11-28T11:10:28.281568+0000","last_undegraded":"2023-11-28T11:10:28.281568+0000","last_fullsized":"2023-11-28T11:10:28.281568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:59.103721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281276+0000","last_change":"2023-11-28T11:10:26.407399+0000","last_active":"2023-11-28T11:10:28.281276+0000","last_peered":"2023-11-28T11:10:28.281276+0000","last_clean":"2023-11-28T11:10:28.281276+0000","last_became_active":"2023-11-28T11:10:26.406541+0000","last_became_peered":"2023-11-28T11:10:26.406541+0000","last_unstale":"2023-11-28T11:10:28.281276+0000","last_undegraded":"2023-11-28T11:10:28.281276+0000","last_fullsized":"2023-11-28T11:10:28.281276+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:36.338136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280387+0000","last_change":"2023-11-28T11:10:26.407515+0000","last_active":"2023-11-28T11:10:28.280387+0000","last_peered":"2023-11-28T11:10:28.280387+0000","last_clean":"2023-11-28T11:10:28.280387+0000","last_became_active":"2023-11-28T11:10:26.407292+0000","last_became_peered":"2023-11-28T11:10:26.407292+0000","last_unstale":"2023-11-28T11:10:28.280387+0000","last_undegraded":"2023-11-28T11:10:28.280387+0000","last_fullsized":"2023-11-28T11:10:28.280387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:15.950069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392574+0000","last_change":"2023-11-28T11:10:26.412162+0000","last_active":"2023-11-28T11:10:27.392574+0000","last_peered":"2023-11-28T11:10:27.392574+0000","last_clean":"2023-11-28T11:10:27.392574+0000","last_became_active":"2023-11-28T11:10:26.411903+0000","last_became_peered":"2023-11-28T11:10:26.411903+0000","last_unstale":"2023-11-28T11:10:27.392574+0000","last_undegraded":"2023-11-28T11:10:27.392574+0000","last_fullsized":"2023-11-28T11:10:27.392574+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:36.553649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281830+0000","last_change":"2023-11-28T11:10:26.411286+0000","last_active":"2023-11-28T11:10:28.281830+0000","last_peered":"2023-11-28T11:10:28.281830+0000","last_clean":"2023-11-28T11:10:28.281830+0000","last_became_active":"2023-11-28T11:10:26.410182+0000","last_became_peered":"2023-11-28T11:10:26.410182+0000","last_unstale":"2023-11-28T11:10:28.281830+0000","last_undegraded":"2023-11-28T11:10:28.281830+0000","last_fullsized":"2023-11-28T11:10:28.281830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:43.887933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.390881+0000","last_change":"2023-11-28T11:10:26.411625+0000","last_active":"2023-11-28T11:10:27.390881+0000","last_peered":"2023-11-28T11:10:27.390881+0000","last_clean":"2023-11-28T11:10:27.390881+0000","last_became_active":"2023-11-28T11:10:26.411381+0000","last_became_peered":"2023-11-28T11:10:26.411381+0000","last_unstale":"2023-11-28T11:10:27.390881+0000","last_undegraded":"2023-11-28T11:10:27.390881+0000","last_fullsized":"2023-11-28T11:10:27.390881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:07.479566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392484+0000","last_change":"2023-11-28T11:10:26.408315+0000","last_active":"2023-11-28T11:10:27.392484+0000","last_peered":"2023-11-28T11:10:27.392484+0000","last_clean":"2023-11-28T11:10:27.392484+0000","last_became_active":"2023-11-28T11:10:26.408047+0000","last_became_peered":"2023-11-28T11:10:26.408047+0000","last_unstale":"2023-11-28T11:10:27.392484+0000","last_undegraded":"2023-11-28T11:10:27.392484+0000","last_fullsized":"2023-11-28T11:10:27.392484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:15.950069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281217+0000","last_change":"2023-11-28T11:10:26.408782+0000","last_active":"2023-11-28T11:10:28.281217+0000","last_peered":"2023-11-28T11:10:28.281217+0000","last_clean":"2023-11-28T11:10:28.281217+0000","last_became_active":"2023-11-28T11:10:26.408335+0000","last_became_peered":"2023-11-28T11:10:26.408335+0000","last_unstale":"2023-11-28T11:10:28.281217+0000","last_undegraded":"2023-11-28T11:10:28.281217+0000","last_fullsized":"2023-11-28T11:10:28.281217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:38.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392328+0000","last_change":"2023-11-28T11:10:26.413688+0000","last_active":"2023-11-28T11:10:27.392328+0000","last_peered":"2023-11-28T11:10:27.392328+0000","last_clean":"2023-11-28T11:10:27.392328+0000","last_became_active":"2023-11-28T11:10:26.413236+0000","last_became_peered":"2023-11-28T11:10:26.413236+0000","last_unstale":"2023-11-28T11:10:27.392328+0000","last_undegraded":"2023-11-28T11:10:27.392328+0000","last_fullsized":"2023-11-28T11:10:27.392328+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:43.456346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281117+0000","last_change":"2023-11-28T11:10:26.414256+0000","last_active":"2023-11-28T11:10:28.281117+0000","last_peered":"2023-11-28T11:10:28.281117+0000","last_clean":"2023-11-28T11:10:28.281117+0000","last_became_active":"2023-11-28T11:10:26.413826+0000","last_became_peered":"2023-11-28T11:10:26.413826+0000","last_unstale":"2023-11-28T11:10:28.281117+0000","last_undegraded":"2023-11-28T11:10:28.281117+0000","last_fullsized":"2023-11-28T11:10:28.281117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:32.874396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282247+0000","last_change":"2023-11-28T11:10:26.407175+0000","last_active":"2023-11-28T11:10:28.282247+0000","last_peered":"2023-11-28T11:10:28.282247+0000","last_clean":"2023-11-28T11:10:28.282247+0000","last_became_active":"2023-11-28T11:10:26.406885+0000","last_became_peered":"2023-11-28T11:10:26.406885+0000","last_unstale":"2023-11-28T11:10:28.282247+0000","last_undegraded":"2023-11-28T11:10:28.282247+0000","last_fullsized":"2023-11-28T11:10:28.282247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:11.919177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.391095+0000","last_change":"2023-11-28T11:10:26.396636+0000","last_active":"2023-11-28T11:10:27.391095+0000","last_peered":"2023-11-28T11:10:27.391095+0000","last_clean":"2023-11-28T11:10:27.391095+0000","last_became_active":"2023-11-28T11:10:26.396360+0000","last_became_peered":"2023-11-28T11:10:26.396360+0000","last_unstale":"2023-11-28T11:10:27.391095+0000","last_undegraded":"2023-11-28T11:10:27.391095+0000","last_fullsized":"2023-11-28T11:10:27.391095+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:02:27.551992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393087+0000","last_change":"2023-11-28T11:10:26.404618+0000","last_active":"2023-11-28T11:10:27.393087+0000","last_peered":"2023-11-28T11:10:27.393087+0000","last_clean":"2023-11-28T11:10:27.393087+0000","last_became_active":"2023-11-28T11:10:26.404318+0000","last_became_peered":"2023-11-28T11:10:26.404318+0000","last_unstale":"2023-11-28T11:10:27.393087+0000","last_undegraded":"2023-11-28T11:10:27.393087+0000","last_fullsized":"2023-11-28T11:10:27.393087+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:32.874396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280645+0000","last_change":"2023-11-28T11:10:26.407922+0000","last_active":"2023-11-28T11:10:28.280645+0000","last_peered":"2023-11-28T11:10:28.280645+0000","last_clean":"2023-11-28T11:10:28.280645+0000","last_became_active":"2023-11-28T11:10:26.407652+0000","last_became_peered":"2023-11-28T11:10:26.407652+0000","last_unstale":"2023-11-28T11:10:28.280645+0000","last_undegraded":"2023-11-28T11:10:28.280645+0000","last_fullsized":"2023-11-28T11:10:28.280645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:28.021018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.390951+0000","last_change":"2023-11-28T11:10:26.403885+0000","last_active":"2023-11-28T11:10:27.390951+0000","last_peered":"2023-11-28T11:10:27.390951+0000","last_clean":"2023-11-28T11:10:27.390951+0000","last_became_active":"2023-11-28T11:10:26.403554+0000","last_became_peered":"2023-11-28T11:10:26.403554+0000","last_unstale":"2023-11-28T11:10:27.390951+0000","last_undegraded":"2023-11-28T11:10:27.390951+0000","last_fullsized":"2023-11-28T11:10:27.390951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:11.919177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392537+0000","last_change":"2023-11-28T11:10:26.413917+0000","last_active":"2023-11-28T11:10:27.392537+0000","last_peered":"2023-11-28T11:10:27.392537+0000","last_clean":"2023-11-28T11:10:27.392537+0000","last_became_active":"2023-11-28T11:10:26.413669+0000","last_became_peered":"2023-11-28T11:10:26.413669+0000","last_unstale":"2023-11-28T11:10:27.392537+0000","last_undegraded":"2023-11-28T11:10:27.392537+0000","last_fullsized":"2023-11-28T11:10:27.392537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:58.033855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282199+0000","last_change":"2023-11-28T11:10:26.401885+0000","last_active":"2023-11-28T11:10:28.282199+0000","last_peered":"2023-11-28T11:10:28.282199+0000","last_clean":"2023-11-28T11:10:28.282199+0000","last_became_active":"2023-11-28T11:10:26.401501+0000","last_became_peered":"2023-11-28T11:10:26.401501+0000","last_unstale":"2023-11-28T11:10:28.282199+0000","last_undegraded":"2023-11-28T11:10:28.282199+0000","last_fullsized":"2023-11-28T11:10:28.282199+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:35.131013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.390776+0000","last_change":"2023-11-28T11:10:26.394472+0000","last_active":"2023-11-28T11:10:27.390776+0000","last_peered":"2023-11-28T11:10:27.390776+0000","last_clean":"2023-11-28T11:10:27.390776+0000","last_became_active":"2023-11-28T11:10:26.394187+0000","last_became_peered":"2023-11-28T11:10:26.394187+0000","last_unstale":"2023-11-28T11:10:27.390776+0000","last_undegraded":"2023-11-28T11:10:27.390776+0000","last_fullsized":"2023-11-28T11:10:27.390776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:39.295444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280460+0000","last_change":"2023-11-28T11:10:26.410125+0000","last_active":"2023-11-28T11:10:28.280460+0000","last_peered":"2023-11-28T11:10:28.280460+0000","last_clean":"2023-11-28T11:10:28.280460+0000","last_became_active":"2023-11-28T11:10:26.409878+0000","last_became_peered":"2023-11-28T11:10:26.409878+0000","last_unstale":"2023-11-28T11:10:28.280460+0000","last_undegraded":"2023-11-28T11:10:28.280460+0000","last_fullsized":"2023-11-28T11:10:28.280460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:47.464212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280772+0000","last_change":"2023-11-28T11:10:26.404993+0000","last_active":"2023-11-28T11:10:28.280772+0000","last_peered":"2023-11-28T11:10:28.280772+0000","last_clean":"2023-11-28T11:10:28.280772+0000","last_became_active":"2023-11-28T11:10:26.404756+0000","last_became_peered":"2023-11-28T11:10:26.404756+0000","last_unstale":"2023-11-28T11:10:28.280772+0000","last_undegraded":"2023-11-28T11:10:28.280772+0000","last_fullsized":"2023-11-28T11:10:28.280772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:01.061849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280929+0000","last_change":"2023-11-28T11:10:25.997798+0000","last_active":"2023-11-28T11:10:28.280929+0000","last_peered":"2023-11-28T11:10:28.280929+0000","last_clean":"2023-11-28T11:10:28.280929+0000","last_became_active":"2023-11-28T11:10:25.997522+0000","last_became_peered":"2023-11-28T11:10:25.997522+0000","last_unstale":"2023-11-28T11:10:28.280929+0000","last_undegraded":"2023-11-28T11:10:28.280929+0000","last_fullsized":"2023-11-28T11:10:28.280929+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:53.065967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491615+0000","last_change":"2023-11-28T11:10:26.405478+0000","last_active":"2023-11-28T11:10:27.491615+0000","last_peered":"2023-11-28T11:10:27.491615+0000","last_clean":"2023-11-28T11:10:27.491615+0000","last_became_active":"2023-11-28T11:10:26.401328+0000","last_became_peered":"2023-11-28T11:10:26.401328+0000","last_unstale":"2023-11-28T11:10:27.491615+0000","last_undegraded":"2023-11-28T11:10:27.491615+0000","last_fullsized":"2023-11-28T11:10:27.491615+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:55.826962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392598+0000","last_change":"2023-11-28T11:10:25.999625+0000","last_active":"2023-11-28T11:10:27.392598+0000","last_peered":"2023-11-28T11:10:27.392598+0000","last_clean":"2023-11-28T11:10:27.392598+0000","last_became_active":"2023-11-28T11:10:25.998910+0000","last_became_peered":"2023-11-28T11:10:25.998910+0000","last_unstale":"2023-11-28T11:10:27.392598+0000","last_undegraded":"2023-11-28T11:10:27.392598+0000","last_fullsized":"2023-11-28T11:10:27.392598+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:35.327593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280664+0000","last_change":"2023-11-28T11:10:26.397086+0000","last_active":"2023-11-28T11:10:28.280664+0000","last_peered":"2023-11-28T11:10:28.280664+0000","last_clean":"2023-11-28T11:10:28.280664+0000","last_became_active":"2023-11-28T11:10:26.396819+0000","last_became_peered":"2023-11-28T11:10:26.396819+0000","last_unstale":"2023-11-28T11:10:28.280664+0000","last_undegraded":"2023-11-28T11:10:28.280664+0000","last_fullsized":"2023-11-28T11:10:28.280664+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:15.404234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280794+0000","last_change":"2023-11-28T11:10:25.996523+0000","last_active":"2023-11-28T11:10:28.280794+0000","last_peered":"2023-11-28T11:10:28.280794+0000","last_clean":"2023-11-28T11:10:28.280794+0000","last_became_active":"2023-11-28T11:10:25.996295+0000","last_became_peered":"2023-11-28T11:10:25.996295+0000","last_unstale":"2023-11-28T11:10:28.280794+0000","last_undegraded":"2023-11-28T11:10:28.280794+0000","last_fullsized":"2023-11-28T11:10:28.280794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:31.863853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280699+0000","last_change":"2023-11-28T11:10:26.403295+0000","last_active":"2023-11-28T11:10:28.280699+0000","last_peered":"2023-11-28T11:10:28.280699+0000","last_clean":"2023-11-28T11:10:28.280699+0000","last_became_active":"2023-11-28T11:10:26.403054+0000","last_became_peered":"2023-11-28T11:10:26.403054+0000","last_unstale":"2023-11-28T11:10:28.280699+0000","last_undegraded":"2023-11-28T11:10:28.280699+0000","last_fullsized":"2023-11-28T11:10:28.280699+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:58.033855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392985+0000","last_change":"2023-11-28T11:10:26.002400+0000","last_active":"2023-11-28T11:10:27.392985+0000","last_peered":"2023-11-28T11:10:27.392985+0000","last_clean":"2023-11-28T11:10:27.392985+0000","last_became_active":"2023-11-28T11:10:26.002005+0000","last_became_peered":"2023-11-28T11:10:26.002005+0000","last_unstale":"2023-11-28T11:10:27.392985+0000","last_undegraded":"2023-11-28T11:10:27.392985+0000","last_fullsized":"2023-11-28T11:10:27.392985+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:20.657127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392374+0000","last_change":"2023-11-28T11:10:26.409087+0000","last_active":"2023-11-28T11:10:27.392374+0000","last_peered":"2023-11-28T11:10:27.392374+0000","last_clean":"2023-11-28T11:10:27.392374+0000","last_became_active":"2023-11-28T11:10:26.408801+0000","last_became_peered":"2023-11-28T11:10:26.408801+0000","last_unstale":"2023-11-28T11:10:27.392374+0000","last_undegraded":"2023-11-28T11:10:27.392374+0000","last_fullsized":"2023-11-28T11:10:27.392374+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:24.728317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281297+0000","last_change":"2023-11-28T11:10:25.999334+0000","last_active":"2023-11-28T11:10:28.281297+0000","last_peered":"2023-11-28T11:10:28.281297+0000","last_clean":"2023-11-28T11:10:28.281297+0000","last_became_active":"2023-11-28T11:10:25.999059+0000","last_became_peered":"2023-11-28T11:10:25.999059+0000","last_unstale":"2023-11-28T11:10:28.281297+0000","last_undegraded":"2023-11-28T11:10:28.281297+0000","last_fullsized":"2023-11-28T11:10:28.281297+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:14.409235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392634+0000","last_change":"2023-11-28T11:10:26.413415+0000","last_active":"2023-11-28T11:10:27.392634+0000","last_peered":"2023-11-28T11:10:27.392634+0000","last_clean":"2023-11-28T11:10:27.392634+0000","last_became_active":"2023-11-28T11:10:26.410411+0000","last_became_peered":"2023-11-28T11:10:26.410411+0000","last_unstale":"2023-11-28T11:10:27.392634+0000","last_undegraded":"2023-11-28T11:10:27.392634+0000","last_fullsized":"2023-11-28T11:10:27.392634+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:47.464212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280476+0000","last_change":"2023-11-28T11:10:26.000932+0000","last_active":"2023-11-28T11:10:28.280476+0000","last_peered":"2023-11-28T11:10:28.280476+0000","last_clean":"2023-11-28T11:10:28.280476+0000","last_became_active":"2023-11-28T11:10:26.000693+0000","last_became_peered":"2023-11-28T11:10:26.000693+0000","last_unstale":"2023-11-28T11:10:28.280476+0000","last_undegraded":"2023-11-28T11:10:28.280476+0000","last_fullsized":"2023-11-28T11:10:28.280476+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:45.187578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282346+0000","last_change":"2023-11-28T11:10:26.411318+0000","last_active":"2023-11-28T11:10:28.282346+0000","last_peered":"2023-11-28T11:10:28.282346+0000","last_clean":"2023-11-28T11:10:28.282346+0000","last_became_active":"2023-11-28T11:10:26.411054+0000","last_became_peered":"2023-11-28T11:10:26.411054+0000","last_unstale":"2023-11-28T11:10:28.282346+0000","last_undegraded":"2023-11-28T11:10:28.282346+0000","last_fullsized":"2023-11-28T11:10:28.282346+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:17.210757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392644+0000","last_change":"2023-11-28T11:10:26.001327+0000","last_active":"2023-11-28T11:10:27.392644+0000","last_peered":"2023-11-28T11:10:27.392644+0000","last_clean":"2023-11-28T11:10:27.392644+0000","last_became_active":"2023-11-28T11:10:26.001002+0000","last_became_peered":"2023-11-28T11:10:26.001002+0000","last_unstale":"2023-11-28T11:10:27.392644+0000","last_undegraded":"2023-11-28T11:10:27.392644+0000","last_fullsized":"2023-11-28T11:10:27.392644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:21.675054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392502+0000","last_change":"2023-11-28T11:10:26.410059+0000","last_active":"2023-11-28T11:10:27.392502+0000","last_peered":"2023-11-28T11:10:27.392502+0000","last_clean":"2023-11-28T11:10:27.392502+0000","last_became_active":"2023-11-28T11:10:26.409692+0000","last_became_peered":"2023-11-28T11:10:26.409692+0000","last_unstale":"2023-11-28T11:10:27.392502+0000","last_undegraded":"2023-11-28T11:10:27.392502+0000","last_fullsized":"2023-11-28T11:10:27.392502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:03.682749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281116+0000","last_change":"2023-11-28T11:10:26.006026+0000","last_active":"2023-11-28T11:10:28.281116+0000","last_peered":"2023-11-28T11:10:28.281116+0000","last_clean":"2023-11-28T11:10:28.281116+0000","last_became_active":"2023-11-28T11:10:26.005788+0000","last_became_peered":"2023-11-28T11:10:26.005788+0000","last_unstale":"2023-11-28T11:10:28.281116+0000","last_undegraded":"2023-11-28T11:10:28.281116+0000","last_fullsized":"2023-11-28T11:10:28.281116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:21.915439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281967+0000","last_change":"2023-11-28T11:10:26.407533+0000","last_active":"2023-11-28T11:10:28.281967+0000","last_peered":"2023-11-28T11:10:28.281967+0000","last_clean":"2023-11-28T11:10:28.281967+0000","last_became_active":"2023-11-28T11:10:26.407224+0000","last_became_peered":"2023-11-28T11:10:26.407224+0000","last_unstale":"2023-11-28T11:10:28.281967+0000","last_undegraded":"2023-11-28T11:10:28.281967+0000","last_fullsized":"2023-11-28T11:10:28.281967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:18.125593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281720+0000","last_change":"2023-11-28T11:10:26.002832+0000","last_active":"2023-11-28T11:10:28.281720+0000","last_peered":"2023-11-28T11:10:28.281720+0000","last_clean":"2023-11-28T11:10:28.281720+0000","last_became_active":"2023-11-28T11:10:26.002538+0000","last_became_peered":"2023-11-28T11:10:26.002538+0000","last_unstale":"2023-11-28T11:10:28.281720+0000","last_undegraded":"2023-11-28T11:10:28.281720+0000","last_fullsized":"2023-11-28T11:10:28.281720+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:58:15.101170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281633+0000","last_change":"2023-11-28T11:10:26.413789+0000","last_active":"2023-11-28T11:10:28.281633+0000","last_peered":"2023-11-28T11:10:28.281633+0000","last_clean":"2023-11-28T11:10:28.281633+0000","last_became_active":"2023-11-28T11:10:26.412314+0000","last_became_peered":"2023-11-28T11:10:26.412314+0000","last_unstale":"2023-11-28T11:10:28.281633+0000","last_undegraded":"2023-11-28T11:10:28.281633+0000","last_fullsized":"2023-11-28T11:10:28.281633+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:17.299342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392651+0000","last_change":"2023-11-28T11:10:26.004872+0000","last_active":"2023-11-28T11:10:27.392651+0000","last_peered":"2023-11-28T11:10:27.392651+0000","last_clean":"2023-11-28T11:10:27.392651+0000","last_became_active":"2023-11-28T11:10:26.004566+0000","last_became_peered":"2023-11-28T11:10:26.004566+0000","last_unstale":"2023-11-28T11:10:27.392651+0000","last_undegraded":"2023-11-28T11:10:27.392651+0000","last_fullsized":"2023-11-28T11:10:27.392651+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:39.013445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491375+0000","last_change":"2023-11-28T11:10:26.402990+0000","last_active":"2023-11-28T11:10:27.491375+0000","last_peered":"2023-11-28T11:10:27.491375+0000","last_clean":"2023-11-28T11:10:27.491375+0000","last_became_active":"2023-11-28T11:10:26.402392+0000","last_became_peered":"2023-11-28T11:10:26.402392+0000","last_unstale":"2023-11-28T11:10:27.491375+0000","last_undegraded":"2023-11-28T11:10:27.491375+0000","last_fullsized":"2023-11-28T11:10:27.491375+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:40.706260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281970+0000","last_change":"2023-11-28T11:10:26.002113+0000","last_active":"2023-11-28T11:10:28.281970+0000","last_peered":"2023-11-28T11:10:28.281970+0000","last_clean":"2023-11-28T11:10:28.281970+0000","last_became_active":"2023-11-28T11:10:26.001825+0000","last_became_peered":"2023-11-28T11:10:26.001825+0000","last_unstale":"2023-11-28T11:10:28.281970+0000","last_undegraded":"2023-11-28T11:10:28.281970+0000","last_fullsized":"2023-11-28T11:10:28.281970+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:04.358989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281685+0000","last_change":"2023-11-28T11:10:26.414776+0000","last_active":"2023-11-28T11:10:28.281685+0000","last_peered":"2023-11-28T11:10:28.281685+0000","last_clean":"2023-11-28T11:10:28.281685+0000","last_became_active":"2023-11-28T11:10:26.414321+0000","last_became_peered":"2023-11-28T11:10:26.414321+0000","last_unstale":"2023-11-28T11:10:28.281685+0000","last_undegraded":"2023-11-28T11:10:28.281685+0000","last_fullsized":"2023-11-28T11:10:28.281685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:34.147447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281533+0000","last_change":"2023-11-28T11:10:25.997372+0000","last_active":"2023-11-28T11:10:28.281533+0000","last_peered":"2023-11-28T11:10:28.281533+0000","last_clean":"2023-11-28T11:10:28.281533+0000","last_became_active":"2023-11-28T11:10:25.997088+0000","last_became_peered":"2023-11-28T11:10:25.997088+0000","last_unstale":"2023-11-28T11:10:28.281533+0000","last_undegraded":"2023-11-28T11:10:28.281533+0000","last_fullsized":"2023-11-28T11:10:28.281533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:35:05.239030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392607+0000","last_change":"2023-11-28T11:10:26.406113+0000","last_active":"2023-11-28T11:10:27.392607+0000","last_peered":"2023-11-28T11:10:27.392607+0000","last_clean":"2023-11-28T11:10:27.392607+0000","last_became_active":"2023-11-28T11:10:26.405825+0000","last_became_peered":"2023-11-28T11:10:26.405825+0000","last_unstale":"2023-11-28T11:10:27.392607+0000","last_undegraded":"2023-11-28T11:10:27.392607+0000","last_fullsized":"2023-11-28T11:10:27.392607+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:51.366529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392762+0000","last_change":"2023-11-28T11:10:26.004985+0000","last_active":"2023-11-28T11:10:27.392762+0000","last_peered":"2023-11-28T11:10:27.392762+0000","last_clean":"2023-11-28T11:10:27.392762+0000","last_became_active":"2023-11-28T11:10:26.004724+0000","last_became_peered":"2023-11-28T11:10:26.004724+0000","last_unstale":"2023-11-28T11:10:27.392762+0000","last_undegraded":"2023-11-28T11:10:27.392762+0000","last_fullsized":"2023-11-28T11:10:27.392762+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:18.695344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281326+0000","last_change":"2023-11-28T11:10:26.410716+0000","last_active":"2023-11-28T11:10:28.281326+0000","last_peered":"2023-11-28T11:10:28.281326+0000","last_clean":"2023-11-28T11:10:28.281326+0000","last_became_active":"2023-11-28T11:10:26.410434+0000","last_became_peered":"2023-11-28T11:10:26.410434+0000","last_unstale":"2023-11-28T11:10:28.281326+0000","last_undegraded":"2023-11-28T11:10:28.281326+0000","last_fullsized":"2023-11-28T11:10:28.281326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:15.419778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282211+0000","last_change":"2023-11-28T11:10:25.998724+0000","last_active":"2023-11-28T11:10:28.282211+0000","last_peered":"2023-11-28T11:10:28.282211+0000","last_clean":"2023-11-28T11:10:28.282211+0000","last_became_active":"2023-11-28T11:10:25.998357+0000","last_became_peered":"2023-11-28T11:10:25.998357+0000","last_unstale":"2023-11-28T11:10:28.282211+0000","last_undegraded":"2023-11-28T11:10:28.282211+0000","last_fullsized":"2023-11-28T11:10:28.282211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:37.166206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281016+0000","last_change":"2023-11-28T11:10:26.413177+0000","last_active":"2023-11-28T11:10:28.281016+0000","last_peered":"2023-11-28T11:10:28.281016+0000","last_clean":"2023-11-28T11:10:28.281016+0000","last_became_active":"2023-11-28T11:10:26.412942+0000","last_became_peered":"2023-11-28T11:10:26.412942+0000","last_unstale":"2023-11-28T11:10:28.281016+0000","last_undegraded":"2023-11-28T11:10:28.281016+0000","last_fullsized":"2023-11-28T11:10:28.281016+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:35.566533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392622+0000","last_change":"2023-11-28T11:10:26.000242+0000","last_active":"2023-11-28T11:10:27.392622+0000","last_peered":"2023-11-28T11:10:27.392622+0000","last_clean":"2023-11-28T11:10:27.392622+0000","last_became_active":"2023-11-28T11:10:25.999986+0000","last_became_peered":"2023-11-28T11:10:25.999986+0000","last_unstale":"2023-11-28T11:10:27.392622+0000","last_undegraded":"2023-11-28T11:10:27.392622+0000","last_fullsized":"2023-11-28T11:10:27.392622+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:56.105652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281225+0000","last_change":"2023-11-28T11:10:26.414086+0000","last_active":"2023-11-28T11:10:28.281225+0000","last_peered":"2023-11-28T11:10:28.281225+0000","last_clean":"2023-11-28T11:10:28.281225+0000","last_became_active":"2023-11-28T11:10:26.412296+0000","last_became_peered":"2023-11-28T11:10:26.412296+0000","last_unstale":"2023-11-28T11:10:28.281225+0000","last_undegraded":"2023-11-28T11:10:28.281225+0000","last_fullsized":"2023-11-28T11:10:28.281225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:54.076510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280962+0000","last_change":"2023-11-28T11:10:26.003726+0000","last_active":"2023-11-28T11:10:28.280962+0000","last_peered":"2023-11-28T11:10:28.280962+0000","last_clean":"2023-11-28T11:10:28.280962+0000","last_became_active":"2023-11-28T11:10:26.003443+0000","last_became_peered":"2023-11-28T11:10:26.003443+0000","last_unstale":"2023-11-28T11:10:28.280962+0000","last_undegraded":"2023-11-28T11:10:28.280962+0000","last_fullsized":"2023-11-28T11:10:28.280962+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:20.503700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393088+0000","last_change":"2023-11-28T11:10:26.401038+0000","last_active":"2023-11-28T11:10:27.393088+0000","last_peered":"2023-11-28T11:10:27.393088+0000","last_clean":"2023-11-28T11:10:27.393088+0000","last_became_active":"2023-11-28T11:10:26.400755+0000","last_became_peered":"2023-11-28T11:10:26.400755+0000","last_unstale":"2023-11-28T11:10:27.393088+0000","last_undegraded":"2023-11-28T11:10:27.393088+0000","last_fullsized":"2023-11-28T11:10:27.393088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:43.887933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491613+0000","last_change":"2023-11-28T11:10:25.999626+0000","last_active":"2023-11-28T11:10:27.491613+0000","last_peered":"2023-11-28T11:10:27.491613+0000","last_clean":"2023-11-28T11:10:27.491613+0000","last_became_active":"2023-11-28T11:10:25.999348+0000","last_became_peered":"2023-11-28T11:10:25.999348+0000","last_unstale":"2023-11-28T11:10:27.491613+0000","last_undegraded":"2023-11-28T11:10:27.491613+0000","last_fullsized":"2023-11-28T11:10:27.491613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:43.906653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393175+0000","last_change":"2023-11-28T11:10:26.402805+0000","last_active":"2023-11-28T11:10:27.393175+0000","last_peered":"2023-11-28T11:10:27.393175+0000","last_clean":"2023-11-28T11:10:27.393175+0000","last_became_active":"2023-11-28T11:10:26.402547+0000","last_became_peered":"2023-11-28T11:10:26.402547+0000","last_unstale":"2023-11-28T11:10:27.393175+0000","last_undegraded":"2023-11-28T11:10:27.393175+0000","last_fullsized":"2023-11-28T11:10:27.393175+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:38.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280877+0000","last_change":"2023-11-28T11:10:25.998272+0000","last_active":"2023-11-28T11:10:28.280877+0000","last_peered":"2023-11-28T11:10:28.280877+0000","last_clean":"2023-11-28T11:10:28.280877+0000","last_became_active":"2023-11-28T11:10:25.998018+0000","last_became_peered":"2023-11-28T11:10:25.998018+0000","last_unstale":"2023-11-28T11:10:28.280877+0000","last_undegraded":"2023-11-28T11:10:28.280877+0000","last_fullsized":"2023-11-28T11:10:28.280877+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:39.013445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491482+0000","last_change":"2023-11-28T11:10:26.403151+0000","last_active":"2023-11-28T11:10:27.491482+0000","last_peered":"2023-11-28T11:10:27.491482+0000","last_clean":"2023-11-28T11:10:27.491482+0000","last_became_active":"2023-11-28T11:10:26.402727+0000","last_became_peered":"2023-11-28T11:10:26.402727+0000","last_unstale":"2023-11-28T11:10:27.491482+0000","last_undegraded":"2023-11-28T11:10:27.491482+0000","last_fullsized":"2023-11-28T11:10:27.491482+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:57.116195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281826+0000","last_change":"2023-11-28T11:10:26.000020+0000","last_active":"2023-11-28T11:10:28.281826+0000","last_peered":"2023-11-28T11:10:28.281826+0000","last_clean":"2023-11-28T11:10:28.281826+0000","last_became_active":"2023-11-28T11:10:25.999724+0000","last_became_peered":"2023-11-28T11:10:25.999724+0000","last_unstale":"2023-11-28T11:10:28.281826+0000","last_undegraded":"2023-11-28T11:10:28.281826+0000","last_fullsized":"2023-11-28T11:10:28.281826+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:15.220689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280764+0000","last_change":"2023-11-28T11:10:26.403140+0000","last_active":"2023-11-28T11:10:28.280764+0000","last_peered":"2023-11-28T11:10:28.280764+0000","last_clean":"2023-11-28T11:10:28.280764+0000","last_became_active":"2023-11-28T11:10:26.402678+0000","last_became_peered":"2023-11-28T11:10:26.402678+0000","last_unstale":"2023-11-28T11:10:28.280764+0000","last_undegraded":"2023-11-28T11:10:28.280764+0000","last_fullsized":"2023-11-28T11:10:28.280764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:18.440998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393094+0000","last_change":"2023-11-28T11:10:25.999027+0000","last_active":"2023-11-28T11:10:27.393094+0000","last_peered":"2023-11-28T11:10:27.393094+0000","last_clean":"2023-11-28T11:10:27.393094+0000","last_became_active":"2023-11-28T11:10:25.998765+0000","last_became_peered":"2023-11-28T11:10:25.998765+0000","last_unstale":"2023-11-28T11:10:27.393094+0000","last_undegraded":"2023-11-28T11:10:27.393094+0000","last_fullsized":"2023-11-28T11:10:27.393094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:54.368325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392427+0000","last_change":"2023-11-28T11:10:26.414745+0000","last_active":"2023-11-28T11:10:27.392427+0000","last_peered":"2023-11-28T11:10:27.392427+0000","last_clean":"2023-11-28T11:10:27.392427+0000","last_became_active":"2023-11-28T11:10:26.413422+0000","last_became_peered":"2023-11-28T11:10:26.413422+0000","last_unstale":"2023-11-28T11:10:27.392427+0000","last_undegraded":"2023-11-28T11:10:27.392427+0000","last_fullsized":"2023-11-28T11:10:27.392427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:50.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491328+0000","last_change":"2023-11-28T11:10:25.993542+0000","last_active":"2023-11-28T11:10:27.491328+0000","last_peered":"2023-11-28T11:10:27.491328+0000","last_clean":"2023-11-28T11:10:27.491328+0000","last_became_active":"2023-11-28T11:10:25.993231+0000","last_became_peered":"2023-11-28T11:10:25.993231+0000","last_unstale":"2023-11-28T11:10:27.491328+0000","last_undegraded":"2023-11-28T11:10:27.491328+0000","last_fullsized":"2023-11-28T11:10:27.491328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:06.469023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281434+0000","last_change":"2023-11-28T11:10:26.413797+0000","last_active":"2023-11-28T11:10:28.281434+0000","last_peered":"2023-11-28T11:10:28.281434+0000","last_clean":"2023-11-28T11:10:28.281434+0000","last_became_active":"2023-11-28T11:10:26.413465+0000","last_became_peered":"2023-11-28T11:10:26.413465+0000","last_unstale":"2023-11-28T11:10:28.281434+0000","last_undegraded":"2023-11-28T11:10:28.281434+0000","last_fullsized":"2023-11-28T11:10:28.281434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:18.125593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281654+0000","last_change":"2023-11-28T11:10:25.997395+0000","last_active":"2023-11-28T11:10:28.281654+0000","last_peered":"2023-11-28T11:10:28.281654+0000","last_clean":"2023-11-28T11:10:28.281654+0000","last_became_active":"2023-11-28T11:10:25.997075+0000","last_became_peered":"2023-11-28T11:10:25.997075+0000","last_unstale":"2023-11-28T11:10:28.281654+0000","last_undegraded":"2023-11-28T11:10:28.281654+0000","last_fullsized":"2023-11-28T11:10:28.281654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:42.877390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491243+0000","last_change":"2023-11-28T11:10:26.407601+0000","last_active":"2023-11-28T11:10:27.491243+0000","last_peered":"2023-11-28T11:10:27.491243+0000","last_clean":"2023-11-28T11:10:27.491243+0000","last_became_active":"2023-11-28T11:10:26.407326+0000","last_became_peered":"2023-11-28T11:10:26.407326+0000","last_unstale":"2023-11-28T11:10:27.491243+0000","last_undegraded":"2023-11-28T11:10:27.491243+0000","last_fullsized":"2023-11-28T11:10:27.491243+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:21.667670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491311+0000","last_change":"2023-11-28T11:10:25.994970+0000","last_active":"2023-11-28T11:10:27.491311+0000","last_peered":"2023-11-28T11:10:27.491311+0000","last_clean":"2023-11-28T11:10:27.491311+0000","last_became_active":"2023-11-28T11:10:25.994492+0000","last_became_peered":"2023-11-28T11:10:25.994492+0000","last_unstale":"2023-11-28T11:10:27.491311+0000","last_undegraded":"2023-11-28T11:10:27.491311+0000","last_fullsized":"2023-11-28T11:10:27.491311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:17.700264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392522+0000","last_change":"2023-11-28T11:10:26.410677+0000","last_active":"2023-11-28T11:10:27.392522+0000","last_peered":"2023-11-28T11:10:27.392522+0000","last_clean":"2023-11-28T11:10:27.392522+0000","last_became_active":"2023-11-28T11:10:26.409353+0000","last_became_peered":"2023-11-28T11:10:26.409353+0000","last_unstale":"2023-11-28T11:10:27.392522+0000","last_undegraded":"2023-11-28T11:10:27.392522+0000","last_fullsized":"2023-11-28T11:10:27.392522+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:27:38.780586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392372+0000","last_change":"2023-11-28T11:10:25.997169+0000","last_active":"2023-11-28T11:10:27.392372+0000","last_peered":"2023-11-28T11:10:27.392372+0000","last_clean":"2023-11-28T11:10:27.392372+0000","last_became_active":"2023-11-28T11:10:25.996863+0000","last_became_peered":"2023-11-28T11:10:25.996863+0000","last_unstale":"2023-11-28T11:10:27.392372+0000","last_undegraded":"2023-11-28T11:10:27.392372+0000","last_fullsized":"2023-11-28T11:10:27.392372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:34.555990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281791+0000","last_change":"2023-11-28T11:10:26.414606+0000","last_active":"2023-11-28T11:10:28.281791+0000","last_peered":"2023-11-28T11:10:28.281791+0000","last_clean":"2023-11-28T11:10:28.281791+0000","last_became_active":"2023-11-28T11:10:26.413966+0000","last_became_peered":"2023-11-28T11:10:26.413966+0000","last_unstale":"2023-11-28T11:10:28.281791+0000","last_undegraded":"2023-11-28T11:10:28.281791+0000","last_fullsized":"2023-11-28T11:10:28.281791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:22.925982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280727+0000","last_change":"2023-11-28T11:10:26.000429+0000","last_active":"2023-11-28T11:10:28.280727+0000","last_peered":"2023-11-28T11:10:28.280727+0000","last_clean":"2023-11-28T11:10:28.280727+0000","last_became_active":"2023-11-28T11:10:26.000087+0000","last_became_peered":"2023-11-28T11:10:26.000087+0000","last_unstale":"2023-11-28T11:10:28.280727+0000","last_undegraded":"2023-11-28T11:10:28.280727+0000","last_fullsized":"2023-11-28T11:10:28.280727+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:25.875992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.400777+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:10:26.400777+0000","last_peered":"2023-11-28T11:10:26.400777+0000","last_clean":"2023-11-28T11:10:26.400777+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:10:26.400777+0000","last_undegraded":"2023-11-28T11:10:26.400777+0000","last_fullsized":"2023-11-28T11:10:26.400777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392629+0000","last_change":"2023-11-28T11:10:26.410088+0000","last_active":"2023-11-28T11:10:27.392629+0000","last_peered":"2023-11-28T11:10:27.392629+0000","last_clean":"2023-11-28T11:10:27.392629+0000","last_became_active":"2023-11-28T11:10:26.409822+0000","last_became_peered":"2023-11-28T11:10:26.409822+0000","last_unstale":"2023-11-28T11:10:27.392629+0000","last_undegraded":"2023-11-28T11:10:27.392629+0000","last_fullsized":"2023-11-28T11:10:27.392629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:25.040675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393067+0000","last_change":"2023-11-28T11:10:25.999971+0000","last_active":"2023-11-28T11:10:27.393067+0000","last_peered":"2023-11-28T11:10:27.393067+0000","last_clean":"2023-11-28T11:10:27.393067+0000","last_became_active":"2023-11-28T11:10:25.999647+0000","last_became_peered":"2023-11-28T11:10:25.999647+0000","last_unstale":"2023-11-28T11:10:27.393067+0000","last_undegraded":"2023-11-28T11:10:27.393067+0000","last_fullsized":"2023-11-28T11:10:27.393067+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:06.628905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.398000+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:10:26.398000+0000","last_peered":"2023-11-28T11:10:26.398000+0000","last_clean":"2023-11-28T11:10:26.398000+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:10:26.398000+0000","last_undegraded":"2023-11-28T11:10:26.398000+0000","last_fullsized":"2023-11-28T11:10:26.398000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280638+0000","last_change":"2023-11-28T11:10:26.408832+0000","last_active":"2023-11-28T11:10:28.280638+0000","last_peered":"2023-11-28T11:10:28.280638+0000","last_clean":"2023-11-28T11:10:28.280638+0000","last_became_active":"2023-11-28T11:10:26.408448+0000","last_became_peered":"2023-11-28T11:10:26.408448+0000","last_unstale":"2023-11-28T11:10:28.280638+0000","last_undegraded":"2023-11-28T11:10:28.280638+0000","last_fullsized":"2023-11-28T11:10:28.280638+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:58.718873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491375+0000","last_change":"2023-11-28T11:10:25.992979+0000","last_active":"2023-11-28T11:10:27.491375+0000","last_peered":"2023-11-28T11:10:27.491375+0000","last_clean":"2023-11-28T11:10:27.491375+0000","last_became_active":"2023-11-28T11:10:25.992711+0000","last_became_peered":"2023-11-28T11:10:25.992711+0000","last_unstale":"2023-11-28T11:10:27.491375+0000","last_undegraded":"2023-11-28T11:10:27.491375+0000","last_fullsized":"2023-11-28T11:10:27.491375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:37.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.388424+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:10:26.388424+0000","last_peered":"2023-11-28T11:10:26.388424+0000","last_clean":"2023-11-28T11:10:26.388424+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:10:26.388424+0000","last_undegraded":"2023-11-28T11:10:26.388424+0000","last_fullsized":"2023-11-28T11:10:26.388424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":187,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.390844+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:10:26.390844+0000","last_peered":"2023-11-28T11:10:26.390844+0000","last_clean":"2023-11-28T11:10:26.390844+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:10:26.390844+0000","last_undegraded":"2023-11-28T11:10:26.390844+0000","last_fullsized":"2023-11-28T11:10:26.390844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392760+0000","last_change":"2023-11-28T11:10:26.402761+0000","last_active":"2023-11-28T11:10:27.392760+0000","last_peered":"2023-11-28T11:10:27.392760+0000","last_clean":"2023-11-28T11:10:27.392760+0000","last_became_active":"2023-11-28T11:10:26.402473+0000","last_became_peered":"2023-11-28T11:10:26.402473+0000","last_unstale":"2023-11-28T11:10:27.392760+0000","last_undegraded":"2023-11-28T11:10:27.392760+0000","last_fullsized":"2023-11-28T11:10:27.392760+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:35.131013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392807+0000","last_change":"2023-11-28T11:10:26.002802+0000","last_active":"2023-11-28T11:10:27.392807+0000","last_peered":"2023-11-28T11:10:27.392807+0000","last_clean":"2023-11-28T11:10:27.392807+0000","last_became_active":"2023-11-28T11:10:26.002540+0000","last_became_peered":"2023-11-28T11:10:26.002540+0000","last_unstale":"2023-11-28T11:10:27.392807+0000","last_undegraded":"2023-11-28T11:10:27.392807+0000","last_fullsized":"2023-11-28T11:10:27.392807+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:58.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.391484+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:10:26.391484+0000","last_peered":"2023-11-28T11:10:26.391484+0000","last_clean":"2023-11-28T11:10:26.391484+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:10:26.391484+0000","last_undegraded":"2023-11-28T11:10:26.391484+0000","last_fullsized":"2023-11-28T11:10:26.391484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392366+0000","last_change":"2023-11-28T11:10:26.408598+0000","last_active":"2023-11-28T11:10:27.392366+0000","last_peered":"2023-11-28T11:10:27.392366+0000","last_clean":"2023-11-28T11:10:27.392366+0000","last_became_active":"2023-11-28T11:10:26.408300+0000","last_became_peered":"2023-11-28T11:10:26.408300+0000","last_unstale":"2023-11-28T11:10:27.392366+0000","last_undegraded":"2023-11-28T11:10:27.392366+0000","last_fullsized":"2023-11-28T11:10:27.392366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:56.623506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282104+0000","last_change":"2023-11-28T11:10:26.000392+0000","last_active":"2023-11-28T11:10:28.282104+0000","last_peered":"2023-11-28T11:10:28.282104+0000","last_clean":"2023-11-28T11:10:28.282104+0000","last_became_active":"2023-11-28T11:10:26.000110+0000","last_became_peered":"2023-11-28T11:10:26.000110+0000","last_unstale":"2023-11-28T11:10:28.282104+0000","last_undegraded":"2023-11-28T11:10:28.282104+0000","last_fullsized":"2023-11-28T11:10:28.282104+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:57.123247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.390737+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:10:26.390737+0000","last_peered":"2023-11-28T11:10:26.390737+0000","last_clean":"2023-11-28T11:10:26.390737+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:10:26.390737+0000","last_undegraded":"2023-11-28T11:10:26.390737+0000","last_fullsized":"2023-11-28T11:10:26.390737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280447+0000","last_change":"2023-11-28T11:10:26.404780+0000","last_active":"2023-11-28T11:10:28.280447+0000","last_peered":"2023-11-28T11:10:28.280447+0000","last_clean":"2023-11-28T11:10:28.280447+0000","last_became_active":"2023-11-28T11:10:26.404521+0000","last_became_peered":"2023-11-28T11:10:26.404521+0000","last_unstale":"2023-11-28T11:10:28.280447+0000","last_undegraded":"2023-11-28T11:10:28.280447+0000","last_fullsized":"2023-11-28T11:10:28.280447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:50.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392560+0000","last_change":"2023-11-28T11:10:26.002760+0000","last_active":"2023-11-28T11:10:27.392560+0000","last_peered":"2023-11-28T11:10:27.392560+0000","last_clean":"2023-11-28T11:10:27.392560+0000","last_became_active":"2023-11-28T11:10:26.002505+0000","last_became_peered":"2023-11-28T11:10:26.002505+0000","last_unstale":"2023-11-28T11:10:27.392560+0000","last_undegraded":"2023-11-28T11:10:27.392560+0000","last_fullsized":"2023-11-28T11:10:27.392560+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:54.816419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.393214+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:10:26.393214+0000","last_peered":"2023-11-28T11:10:26.393214+0000","last_clean":"2023-11-28T11:10:26.393214+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:10:26.393214+0000","last_undegraded":"2023-11-28T11:10:26.393214+0000","last_fullsized":"2023-11-28T11:10:26.393214+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281887+0000","last_change":"2023-11-28T11:10:26.409836+0000","last_active":"2023-11-28T11:10:28.281887+0000","last_peered":"2023-11-28T11:10:28.281887+0000","last_clean":"2023-11-28T11:10:28.281887+0000","last_became_active":"2023-11-28T11:10:26.405152+0000","last_became_peered":"2023-11-28T11:10:26.405152+0000","last_unstale":"2023-11-28T11:10:28.281887+0000","last_undegraded":"2023-11-28T11:10:28.281887+0000","last_fullsized":"2023-11-28T11:10:28.281887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:51.366529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392662+0000","last_change":"2023-11-28T11:10:26.005110+0000","last_active":"2023-11-28T11:10:27.392662+0000","last_peered":"2023-11-28T11:10:27.392662+0000","last_clean":"2023-11-28T11:10:27.392662+0000","last_became_active":"2023-11-28T11:10:26.004815+0000","last_became_peered":"2023-11-28T11:10:26.004815+0000","last_unstale":"2023-11-28T11:10:27.392662+0000","last_undegraded":"2023-11-28T11:10:27.392662+0000","last_fullsized":"2023-11-28T11:10:27.392662+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:45.187578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.396773+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:10:26.396773+0000","last_peered":"2023-11-28T11:10:26.396773+0000","last_clean":"2023-11-28T11:10:26.396773+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:10:26.396773+0000","last_undegraded":"2023-11-28T11:10:26.396773+0000","last_fullsized":"2023-11-28T11:10:26.396773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393105+0000","last_change":"2023-11-28T11:10:26.406365+0000","last_active":"2023-11-28T11:10:27.393105+0000","last_peered":"2023-11-28T11:10:27.393105+0000","last_clean":"2023-11-28T11:10:27.393105+0000","last_became_active":"2023-11-28T11:10:26.406065+0000","last_became_peered":"2023-11-28T11:10:26.406065+0000","last_unstale":"2023-11-28T11:10:27.393105+0000","last_undegraded":"2023-11-28T11:10:27.393105+0000","last_fullsized":"2023-11-28T11:10:27.393105+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:34.147447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280807+0000","last_change":"2023-11-28T11:10:25.994288+0000","last_active":"2023-11-28T11:10:28.280807+0000","last_peered":"2023-11-28T11:10:28.280807+0000","last_clean":"2023-11-28T11:10:28.280807+0000","last_became_active":"2023-11-28T11:10:25.994000+0000","last_became_peered":"2023-11-28T11:10:25.994000+0000","last_unstale":"2023-11-28T11:10:28.280807+0000","last_undegraded":"2023-11-28T11:10:28.280807+0000","last_fullsized":"2023-11-28T11:10:28.280807+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:17.115051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.399592+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:10:26.399592+0000","last_peered":"2023-11-28T11:10:26.399592+0000","last_clean":"2023-11-28T11:10:26.399592+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:10:26.399592+0000","last_undegraded":"2023-11-28T11:10:26.399592+0000","last_fullsized":"2023-11-28T11:10:26.399592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10276,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361564,"statfs":{"total":96636764160,"available":96626241536,"internally_reserved":0,"allocated":1208320,"data_stored":870716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.57999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39400000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.501}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.621}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9032,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362808,"statfs":{"total":96636764160,"available":96627515392,"internally_reserved":0,"allocated":589824,"data_stored":271854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.629},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.67800000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.65000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.504}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.61499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10208,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361632,"statfs":{"total":96636764160,"available":96626311168,"internally_reserved":0,"allocated":1204224,"data_stored":870646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9240,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362600,"statfs":{"total":96636764160,"available":96627302400,"internally_reserved":0,"allocated":606208,"data_stored":273968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.27000000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.34300000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.6429999999999998,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.73399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.50800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9056,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362784,"statfs":{"total":96636764160,"available":96627490816,"internally_reserved":0,"allocated":614400,"data_stored":280244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.309}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.50900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9248,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362592,"statfs":{"total":96636764160,"available":96627294208,"internally_reserved":0,"allocated":614400,"data_stored":280244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.133,"5min":5.133,"15min":5.133},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9252,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362588,"statfs":{"total":96636764160,"available":96627290112,"internally_reserved":0,"allocated":618496,"data_stored":281736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.64700000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57099999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.378},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.44900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42999999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.505}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9120,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362720,"statfs":{"total":96636764160,"available":96627425280,"internally_reserved":0,"allocated":614400,"data_stored":280824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":8.4610000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":8.4830000000000005,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.601,"5min":6.601,"15min":6.601},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.627},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.45700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:30.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:30.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.613+0000 7f1bc3fa0700 1 Processor -- start 2023-11-28T11:10:30.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.613+0000 7f1bc3fa0700 1 -- start start 2023-11-28T11:10:30.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc3fa0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1bbc15a8e0 0x7f1bbc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc3fa0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1bbc0b2ea0 0x7f1bbc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc3fa0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bbc0b32d0 con 0x7f1bbc15a510 2023-11-28T11:10:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc3fa0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc0cd780 con 0x7f1bbc15a8e0 2023-11-28T11:10:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc3fa0700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc15f1d0 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1bbc15a8e0 0x7f1bbc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1bbc15a8e0 0x7f1bbc0b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51726/0 (socket says 172.21.15.83:51726) 2023-11-28T11:10:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 -- 172.21.15.83:0/1233806829 learned_addr learned my addr 172.21.15.83:0/1233806829 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc253d700 1 --2- 172.21.15.83:0/1233806829 >> v2:172.21.15.90:3300/0 conn(0x7f1bbc0b2ea0 0x7f1bbc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 -- 172.21.15.83:0/1233806829 >> v2:172.21.15.90:3300/0 conn(0x7f1bbc0b2ea0 msgr2=0x7f1bbc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 --2- 172.21.15.83:0/1233806829 >> v2:172.21.15.90:3300/0 conn(0x7f1bbc0b2ea0 0x7f1bbc0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 -- 172.21.15.83:0/1233806829 >> v1:172.21.15.90:6789/0 conn(0x7f1bbc15a510 legacy=0x7f1bbc0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.614+0000 7f1bc153b700 1 -- 172.21.15.83:0/1233806829 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc15eef0 con 0x7f1bbc15a8e0 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc153b700 1 --2- 172.21.15.83:0/1233806829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc15a8e0 0x7f1bbc0b2960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0718c0 tx=0x7f1bb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9799e9fb953bbc3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc0d3a700 1 -- 172.21.15.83:0/1233806829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8004230 con 0x7f1bbc15a8e0 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc0d3a700 1 -- 172.21.15.83:0/1233806829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8004390 con 0x7f1bbc15a8e0 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc0d3a700 1 -- 172.21.15.83:0/1233806829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80036a0 con 0x7f1bbc15a8e0 2023-11-28T11:10:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/1233806829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc15a8e0 msgr2=0x7f1bbc0b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/1233806829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc15a8e0 0x7f1bbc0b2960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0718c0 tx=0x7f1bb8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/1233806829 shutdown_connections 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/1233806829 >> v2:172.21.15.90:3300/0 conn(0x7f1bbc0b2ea0 0x7f1bbc0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/1233806829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc15a8e0 0x7f1bbc0b2960 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/1233806829 >> 172.21.15.83:0/1233806829 conn(0x7f1bbc0cd960 msgr2=0x7f1bbc0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/1233806829 shutdown_connections 2023-11-28T11:10:30.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.615+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/1233806829 wait complete. 2023-11-28T11:10:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.616+0000 7f1bc3fa0700 1 Processor -- start 2023-11-28T11:10:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 -- start start 2023-11-28T11:10:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 0x7f1bbc1f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bbc15a510 0x7f1bbc0c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1bbc15a8e0 0x7f1bbc0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1f4310 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1efd80 con 0x7f1bbc15a510 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc3fa0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc0c95d0 con 0x7f1bbc15a8e0 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc1d3c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 0x7f1bbc1f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc1d3c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 0x7f1bbc1f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51730/0 (socket says 172.21.15.83:51730) 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc1d3c700 1 -- 172.21.15.83:0/4240212381 learned_addr learned my addr 172.21.15.83:0/4240212381 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc153b700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bbc15a510 0x7f1bbc0c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.617+0000 7f1bc253d700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1bbc15a8e0 0x7f1bbc0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc1d3c700 1 -- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1bbc15a8e0 msgr2=0x7f1bbc0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc1d3c700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1bbc15a8e0 0x7f1bbc0c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc1d3c700 1 -- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bbc15a510 msgr2=0x7f1bbc0c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc1d3c700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bbc15a510 0x7f1bbc0c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc1d3c700 1 -- 172.21.15.83:0/4240212381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8009550 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc253d700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1bbc15a8e0 0x7f1bbc0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:30.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc1d3c700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 0x7f1bbc1f3370 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00c4d0 tx=0x7f1bac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac011790 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac0118f0 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc0cf0b0 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac011ca0 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.618+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc0cedd0 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.619+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc0cf390 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.619+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1bac010050 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.619+0000 7f1bb27fc700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ba00682f0 0x7f1ba006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.620+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1bac085b50 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.620+0000 7f1bc153b700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ba00682f0 0x7f1ba006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.620+0000 7f1bc153b700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ba00682f0 0x7f1ba006a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009130 tx=0x7f1bb8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.624+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1bac052270 con 0x7f1bbc0b2ea0 2023-11-28T11:10:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.800+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bbc074320 con 0x7f1ba00682f0 2023-11-28T11:10:30.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.808+0000 7f1bb27fc700 1 -- 172.21.15.83:0/4240212381 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311366 (secure 0 0 0) 0x7f1bbc074320 con 0x7f1ba00682f0 2023-11-28T11:10:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ba00682f0 msgr2=0x7f1ba006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ba00682f0 0x7f1ba006a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009130 tx=0x7f1bb8008030 comp rx=0 tx=0).stop 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 msgr2=0x7f1bbc1f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 0x7f1bbc1f3370 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00c4d0 tx=0x7f1bac00c890 comp rx=0 tx=0).stop 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 shutdown_connections 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1bbc15a8e0 0x7f1bbc0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1ba00682f0 0x7f1ba006a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bbc0b2ea0 0x7f1bbc1f3370 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 --2- 172.21.15.83:0/4240212381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1bbc15a510 0x7f1bbc0c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 >> 172.21.15.83:0/4240212381 conn(0x7f1bbc0cd960 msgr2=0x7f1bbc0b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 shutdown_connections 2023-11-28T11:10:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.811+0000 7f1bc3fa0700 1 -- 172.21.15.83:0/4240212381 wait complete. 2023-11-28T11:10:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:10:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-11-28T11:10:29.592276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":75432,"kb_used_data":5928,"kb_used_omap":0,"kb_used_meta":69440,"kb_avail":754899288,"statfs":{"total":773094113280,"available":773016870912,"internally_reserved":0,"allocated":6070272,"data_stored":3410232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71106560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.620219"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.391193+0000","last_change":"2023-11-28T11:10:26.414517+0000","last_active":"2023-11-28T11:10:27.391193+0000","last_peered":"2023-11-28T11:10:27.391193+0000","last_clean":"2023-11-28T11:10:27.391193+0000","last_became_active":"2023-11-28T11:10:26.414258+0000","last_became_peered":"2023-11-28T11:10:26.414258+0000","last_unstale":"2023-11-28T11:10:27.391193+0000","last_undegraded":"2023-11-28T11:10:27.391193+0000","last_fullsized":"2023-11-28T11:10:27.391193+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:21.514243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392499+0000","last_change":"2023-11-28T11:10:26.414992+0000","last_active":"2023-11-28T11:10:27.392499+0000","last_peered":"2023-11-28T11:10:27.392499+0000","last_clean":"2023-11-28T11:10:27.392499+0000","last_became_active":"2023-11-28T11:10:26.413263+0000","last_became_peered":"2023-11-28T11:10:26.413263+0000","last_unstale":"2023-11-28T11:10:27.392499+0000","last_undegraded":"2023-11-28T11:10:27.392499+0000","last_fullsized":"2023-11-28T11:10:27.392499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:11.158839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281564+0000","last_change":"2023-11-28T11:10:26.404834+0000","last_active":"2023-11-28T11:10:28.281564+0000","last_peered":"2023-11-28T11:10:28.281564+0000","last_clean":"2023-11-28T11:10:28.281564+0000","last_became_active":"2023-11-28T11:10:26.404563+0000","last_became_peered":"2023-11-28T11:10:26.404563+0000","last_unstale":"2023-11-28T11:10:28.281564+0000","last_undegraded":"2023-11-28T11:10:28.281564+0000","last_fullsized":"2023-11-28T11:10:28.281564+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:26.886535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392916+0000","last_change":"2023-11-28T11:10:26.405022+0000","last_active":"2023-11-28T11:10:27.392916+0000","last_peered":"2023-11-28T11:10:27.392916+0000","last_clean":"2023-11-28T11:10:27.392916+0000","last_became_active":"2023-11-28T11:10:26.402208+0000","last_became_peered":"2023-11-28T11:10:26.402208+0000","last_unstale":"2023-11-28T11:10:27.392916+0000","last_undegraded":"2023-11-28T11:10:27.392916+0000","last_fullsized":"2023-11-28T11:10:27.392916+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:26.886535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.391215+0000","last_change":"2023-11-28T11:10:26.395758+0000","last_active":"2023-11-28T11:10:27.391215+0000","last_peered":"2023-11-28T11:10:27.391215+0000","last_clean":"2023-11-28T11:10:27.391215+0000","last_became_active":"2023-11-28T11:10:26.395449+0000","last_became_peered":"2023-11-28T11:10:26.395449+0000","last_unstale":"2023-11-28T11:10:27.391215+0000","last_undegraded":"2023-11-28T11:10:27.391215+0000","last_fullsized":"2023-11-28T11:10:27.391215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:17.210757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280468+0000","last_change":"2023-11-28T11:10:26.397024+0000","last_active":"2023-11-28T11:10:28.280468+0000","last_peered":"2023-11-28T11:10:28.280468+0000","last_clean":"2023-11-28T11:10:28.280468+0000","last_became_active":"2023-11-28T11:10:26.396792+0000","last_became_peered":"2023-11-28T11:10:26.396792+0000","last_unstale":"2023-11-28T11:10:28.280468+0000","last_undegraded":"2023-11-28T11:10:28.280468+0000","last_fullsized":"2023-11-28T11:10:28.280468+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:27.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281161+0000","last_change":"2023-11-28T11:10:26.413044+0000","last_active":"2023-11-28T11:10:28.281161+0000","last_peered":"2023-11-28T11:10:28.281161+0000","last_clean":"2023-11-28T11:10:28.281161+0000","last_became_active":"2023-11-28T11:10:26.412783+0000","last_became_peered":"2023-11-28T11:10:26.412783+0000","last_unstale":"2023-11-28T11:10:28.281161+0000","last_undegraded":"2023-11-28T11:10:28.281161+0000","last_fullsized":"2023-11-28T11:10:28.281161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:19.705887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392284+0000","last_change":"2023-11-28T11:10:26.409134+0000","last_active":"2023-11-28T11:10:27.392284+0000","last_peered":"2023-11-28T11:10:27.392284+0000","last_clean":"2023-11-28T11:10:27.392284+0000","last_became_active":"2023-11-28T11:10:26.408858+0000","last_became_peered":"2023-11-28T11:10:26.408858+0000","last_unstale":"2023-11-28T11:10:27.392284+0000","last_undegraded":"2023-11-28T11:10:27.392284+0000","last_fullsized":"2023-11-28T11:10:27.392284+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:17.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282084+0000","last_change":"2023-11-28T11:10:26.401800+0000","last_active":"2023-11-28T11:10:28.282084+0000","last_peered":"2023-11-28T11:10:28.282084+0000","last_clean":"2023-11-28T11:10:28.282084+0000","last_became_active":"2023-11-28T11:10:26.401523+0000","last_became_peered":"2023-11-28T11:10:26.401523+0000","last_unstale":"2023-11-28T11:10:28.282084+0000","last_undegraded":"2023-11-28T11:10:28.282084+0000","last_fullsized":"2023-11-28T11:10:28.282084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:46.198121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491525+0000","last_change":"2023-11-28T11:10:26.406955+0000","last_active":"2023-11-28T11:10:27.491525+0000","last_peered":"2023-11-28T11:10:27.491525+0000","last_clean":"2023-11-28T11:10:27.491525+0000","last_became_active":"2023-11-28T11:10:26.406556+0000","last_became_peered":"2023-11-28T11:10:26.406556+0000","last_unstale":"2023-11-28T11:10:27.491525+0000","last_undegraded":"2023-11-28T11:10:27.491525+0000","last_fullsized":"2023-11-28T11:10:27.491525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:07.639448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491523+0000","last_change":"2023-11-28T11:10:26.410342+0000","last_active":"2023-11-28T11:10:27.491523+0000","last_peered":"2023-11-28T11:10:27.491523+0000","last_clean":"2023-11-28T11:10:27.491523+0000","last_became_active":"2023-11-28T11:10:26.410076+0000","last_became_peered":"2023-11-28T11:10:26.410076+0000","last_unstale":"2023-11-28T11:10:27.491523+0000","last_undegraded":"2023-11-28T11:10:27.491523+0000","last_fullsized":"2023-11-28T11:10:27.491523+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:22.685597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393090+0000","last_change":"2023-11-28T11:10:26.405323+0000","last_active":"2023-11-28T11:10:27.393090+0000","last_peered":"2023-11-28T11:10:27.393090+0000","last_clean":"2023-11-28T11:10:27.393090+0000","last_became_active":"2023-11-28T11:10:26.404636+0000","last_became_peered":"2023-11-28T11:10:26.404636+0000","last_unstale":"2023-11-28T11:10:27.393090+0000","last_undegraded":"2023-11-28T11:10:27.393090+0000","last_fullsized":"2023-11-28T11:10:27.393090+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:54.076510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282359+0000","last_change":"2023-11-28T11:10:26.407794+0000","last_active":"2023-11-28T11:10:28.282359+0000","last_peered":"2023-11-28T11:10:28.282359+0000","last_clean":"2023-11-28T11:10:28.282359+0000","last_became_active":"2023-11-28T11:10:26.407527+0000","last_became_peered":"2023-11-28T11:10:26.407527+0000","last_unstale":"2023-11-28T11:10:28.282359+0000","last_undegraded":"2023-11-28T11:10:28.282359+0000","last_fullsized":"2023-11-28T11:10:28.282359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:39.295444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281568+0000","last_change":"2023-11-28T11:10:26.404941+0000","last_active":"2023-11-28T11:10:28.281568+0000","last_peered":"2023-11-28T11:10:28.281568+0000","last_clean":"2023-11-28T11:10:28.281568+0000","last_became_active":"2023-11-28T11:10:26.404641+0000","last_became_peered":"2023-11-28T11:10:26.404641+0000","last_unstale":"2023-11-28T11:10:28.281568+0000","last_undegraded":"2023-11-28T11:10:28.281568+0000","last_fullsized":"2023-11-28T11:10:28.281568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:59.103721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281276+0000","last_change":"2023-11-28T11:10:26.407399+0000","last_active":"2023-11-28T11:10:28.281276+0000","last_peered":"2023-11-28T11:10:28.281276+0000","last_clean":"2023-11-28T11:10:28.281276+0000","last_became_active":"2023-11-28T11:10:26.406541+0000","last_became_peered":"2023-11-28T11:10:26.406541+0000","last_unstale":"2023-11-28T11:10:28.281276+0000","last_undegraded":"2023-11-28T11:10:28.281276+0000","last_fullsized":"2023-11-28T11:10:28.281276+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:36.338136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280387+0000","last_change":"2023-11-28T11:10:26.407515+0000","last_active":"2023-11-28T11:10:28.280387+0000","last_peered":"2023-11-28T11:10:28.280387+0000","last_clean":"2023-11-28T11:10:28.280387+0000","last_became_active":"2023-11-28T11:10:26.407292+0000","last_became_peered":"2023-11-28T11:10:26.407292+0000","last_unstale":"2023-11-28T11:10:28.280387+0000","last_undegraded":"2023-11-28T11:10:28.280387+0000","last_fullsized":"2023-11-28T11:10:28.280387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:15.950069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392574+0000","last_change":"2023-11-28T11:10:26.412162+0000","last_active":"2023-11-28T11:10:27.392574+0000","last_peered":"2023-11-28T11:10:27.392574+0000","last_clean":"2023-11-28T11:10:27.392574+0000","last_became_active":"2023-11-28T11:10:26.411903+0000","last_became_peered":"2023-11-28T11:10:26.411903+0000","last_unstale":"2023-11-28T11:10:27.392574+0000","last_undegraded":"2023-11-28T11:10:27.392574+0000","last_fullsized":"2023-11-28T11:10:27.392574+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:36.553649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281830+0000","last_change":"2023-11-28T11:10:26.411286+0000","last_active":"2023-11-28T11:10:28.281830+0000","last_peered":"2023-11-28T11:10:28.281830+0000","last_clean":"2023-11-28T11:10:28.281830+0000","last_became_active":"2023-11-28T11:10:26.410182+0000","last_became_peered":"2023-11-28T11:10:26.410182+0000","last_unstale":"2023-11-28T11:10:28.281830+0000","last_undegraded":"2023-11-28T11:10:28.281830+0000","last_fullsized":"2023-11-28T11:10:28.281830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:43.887933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.390881+0000","last_change":"2023-11-28T11:10:26.411625+0000","last_active":"2023-11-28T11:10:27.390881+0000","last_peered":"2023-11-28T11:10:27.390881+0000","last_clean":"2023-11-28T11:10:27.390881+0000","last_became_active":"2023-11-28T11:10:26.411381+0000","last_became_peered":"2023-11-28T11:10:26.411381+0000","last_unstale":"2023-11-28T11:10:27.390881+0000","last_undegraded":"2023-11-28T11:10:27.390881+0000","last_fullsized":"2023-11-28T11:10:27.390881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:07.479566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392484+0000","last_change":"2023-11-28T11:10:26.408315+0000","last_active":"2023-11-28T11:10:27.392484+0000","last_peered":"2023-11-28T11:10:27.392484+0000","last_clean":"2023-11-28T11:10:27.392484+0000","last_became_active":"2023-11-28T11:10:26.408047+0000","last_became_peered":"2023-11-28T11:10:26.408047+0000","last_unstale":"2023-11-28T11:10:27.392484+0000","last_undegraded":"2023-11-28T11:10:27.392484+0000","last_fullsized":"2023-11-28T11:10:27.392484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:15.950069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281217+0000","last_change":"2023-11-28T11:10:26.408782+0000","last_active":"2023-11-28T11:10:28.281217+0000","last_peered":"2023-11-28T11:10:28.281217+0000","last_clean":"2023-11-28T11:10:28.281217+0000","last_became_active":"2023-11-28T11:10:26.408335+0000","last_became_peered":"2023-11-28T11:10:26.408335+0000","last_unstale":"2023-11-28T11:10:28.281217+0000","last_undegraded":"2023-11-28T11:10:28.281217+0000","last_fullsized":"2023-11-28T11:10:28.281217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:38.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392328+0000","last_change":"2023-11-28T11:10:26.413688+0000","last_active":"2023-11-28T11:10:27.392328+0000","last_peered":"2023-11-28T11:10:27.392328+0000","last_clean":"2023-11-28T11:10:27.392328+0000","last_became_active":"2023-11-28T11:10:26.413236+0000","last_became_peered":"2023-11-28T11:10:26.413236+0000","last_unstale":"2023-11-28T11:10:27.392328+0000","last_undegraded":"2023-11-28T11:10:27.392328+0000","last_fullsized":"2023-11-28T11:10:27.392328+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:43.456346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281117+0000","last_change":"2023-11-28T11:10:26.414256+0000","last_active":"2023-11-28T11:10:28.281117+0000","last_peered":"2023-11-28T11:10:28.281117+0000","last_clean":"2023-11-28T11:10:28.281117+0000","last_became_active":"2023-11-28T11:10:26.413826+0000","last_became_peered":"2023-11-28T11:10:26.413826+0000","last_unstale":"2023-11-28T11:10:28.281117+0000","last_undegraded":"2023-11-28T11:10:28.281117+0000","last_fullsized":"2023-11-28T11:10:28.281117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:32.874396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282247+0000","last_change":"2023-11-28T11:10:26.407175+0000","last_active":"2023-11-28T11:10:28.282247+0000","last_peered":"2023-11-28T11:10:28.282247+0000","last_clean":"2023-11-28T11:10:28.282247+0000","last_became_active":"2023-11-28T11:10:26.406885+0000","last_became_peered":"2023-11-28T11:10:26.406885+0000","last_unstale":"2023-11-28T11:10:28.282247+0000","last_undegraded":"2023-11-28T11:10:28.282247+0000","last_fullsized":"2023-11-28T11:10:28.282247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:11.919177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.391095+0000","last_change":"2023-11-28T11:10:26.396636+0000","last_active":"2023-11-28T11:10:27.391095+0000","last_peered":"2023-11-28T11:10:27.391095+0000","last_clean":"2023-11-28T11:10:27.391095+0000","last_became_active":"2023-11-28T11:10:26.396360+0000","last_became_peered":"2023-11-28T11:10:26.396360+0000","last_unstale":"2023-11-28T11:10:27.391095+0000","last_undegraded":"2023-11-28T11:10:27.391095+0000","last_fullsized":"2023-11-28T11:10:27.391095+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:02:27.551992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393087+0000","last_change":"2023-11-28T11:10:26.404618+0000","last_active":"2023-11-28T11:10:27.393087+0000","last_peered":"2023-11-28T11:10:27.393087+0000","last_clean":"2023-11-28T11:10:27.393087+0000","last_became_active":"2023-11-28T11:10:26.404318+0000","last_became_peered":"2023-11-28T11:10:26.404318+0000","last_unstale":"2023-11-28T11:10:27.393087+0000","last_undegraded":"2023-11-28T11:10:27.393087+0000","last_fullsized":"2023-11-28T11:10:27.393087+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:32.874396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280645+0000","last_change":"2023-11-28T11:10:26.407922+0000","last_active":"2023-11-28T11:10:28.280645+0000","last_peered":"2023-11-28T11:10:28.280645+0000","last_clean":"2023-11-28T11:10:28.280645+0000","last_became_active":"2023-11-28T11:10:26.407652+0000","last_became_peered":"2023-11-28T11:10:26.407652+0000","last_unstale":"2023-11-28T11:10:28.280645+0000","last_undegraded":"2023-11-28T11:10:28.280645+0000","last_fullsized":"2023-11-28T11:10:28.280645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:28.021018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.390951+0000","last_change":"2023-11-28T11:10:26.403885+0000","last_active":"2023-11-28T11:10:27.390951+0000","last_peered":"2023-11-28T11:10:27.390951+0000","last_clean":"2023-11-28T11:10:27.390951+0000","last_became_active":"2023-11-28T11:10:26.403554+0000","last_became_peered":"2023-11-28T11:10:26.403554+0000","last_unstale":"2023-11-28T11:10:27.390951+0000","last_undegraded":"2023-11-28T11:10:27.390951+0000","last_fullsized":"2023-11-28T11:10:27.390951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:11.919177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392537+0000","last_change":"2023-11-28T11:10:26.413917+0000","last_active":"2023-11-28T11:10:27.392537+0000","last_peered":"2023-11-28T11:10:27.392537+0000","last_clean":"2023-11-28T11:10:27.392537+0000","last_became_active":"2023-11-28T11:10:26.413669+0000","last_became_peered":"2023-11-28T11:10:26.413669+0000","last_unstale":"2023-11-28T11:10:27.392537+0000","last_undegraded":"2023-11-28T11:10:27.392537+0000","last_fullsized":"2023-11-28T11:10:27.392537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:58.033855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282199+0000","last_change":"2023-11-28T11:10:26.401885+0000","last_active":"2023-11-28T11:10:28.282199+0000","last_peered":"2023-11-28T11:10:28.282199+0000","last_clean":"2023-11-28T11:10:28.282199+0000","last_became_active":"2023-11-28T11:10:26.401501+0000","last_became_peered":"2023-11-28T11:10:26.401501+0000","last_unstale":"2023-11-28T11:10:28.282199+0000","last_undegraded":"2023-11-28T11:10:28.282199+0000","last_fullsized":"2023-11-28T11:10:28.282199+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:35.131013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.390776+0000","last_change":"2023-11-28T11:10:26.394472+0000","last_active":"2023-11-28T11:10:27.390776+0000","last_peered":"2023-11-28T11:10:27.390776+0000","last_clean":"2023-11-28T11:10:27.390776+0000","last_became_active":"2023-11-28T11:10:26.394187+0000","last_became_peered":"2023-11-28T11:10:26.394187+0000","last_unstale":"2023-11-28T11:10:27.390776+0000","last_undegraded":"2023-11-28T11:10:27.390776+0000","last_fullsized":"2023-11-28T11:10:27.390776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:39.295444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280460+0000","last_change":"2023-11-28T11:10:26.410125+0000","last_active":"2023-11-28T11:10:28.280460+0000","last_peered":"2023-11-28T11:10:28.280460+0000","last_clean":"2023-11-28T11:10:28.280460+0000","last_became_active":"2023-11-28T11:10:26.409878+0000","last_became_peered":"2023-11-28T11:10:26.409878+0000","last_unstale":"2023-11-28T11:10:28.280460+0000","last_undegraded":"2023-11-28T11:10:28.280460+0000","last_fullsized":"2023-11-28T11:10:28.280460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:47.464212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280772+0000","last_change":"2023-11-28T11:10:26.404993+0000","last_active":"2023-11-28T11:10:28.280772+0000","last_peered":"2023-11-28T11:10:28.280772+0000","last_clean":"2023-11-28T11:10:28.280772+0000","last_became_active":"2023-11-28T11:10:26.404756+0000","last_became_peered":"2023-11-28T11:10:26.404756+0000","last_unstale":"2023-11-28T11:10:28.280772+0000","last_undegraded":"2023-11-28T11:10:28.280772+0000","last_fullsized":"2023-11-28T11:10:28.280772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:01.061849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280929+0000","last_change":"2023-11-28T11:10:25.997798+0000","last_active":"2023-11-28T11:10:28.280929+0000","last_peered":"2023-11-28T11:10:28.280929+0000","last_clean":"2023-11-28T11:10:28.280929+0000","last_became_active":"2023-11-28T11:10:25.997522+0000","last_became_peered":"2023-11-28T11:10:25.997522+0000","last_unstale":"2023-11-28T11:10:28.280929+0000","last_undegraded":"2023-11-28T11:10:28.280929+0000","last_fullsized":"2023-11-28T11:10:28.280929+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:53.065967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491615+0000","last_change":"2023-11-28T11:10:26.405478+0000","last_active":"2023-11-28T11:10:27.491615+0000","last_peered":"2023-11-28T11:10:27.491615+0000","last_clean":"2023-11-28T11:10:27.491615+0000","last_became_active":"2023-11-28T11:10:26.401328+0000","last_became_peered":"2023-11-28T11:10:26.401328+0000","last_unstale":"2023-11-28T11:10:27.491615+0000","last_undegraded":"2023-11-28T11:10:27.491615+0000","last_fullsized":"2023-11-28T11:10:27.491615+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:55.826962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392598+0000","last_change":"2023-11-28T11:10:25.999625+0000","last_active":"2023-11-28T11:10:27.392598+0000","last_peered":"2023-11-28T11:10:27.392598+0000","last_clean":"2023-11-28T11:10:27.392598+0000","last_became_active":"2023-11-28T11:10:25.998910+0000","last_became_peered":"2023-11-28T11:10:25.998910+0000","last_unstale":"2023-11-28T11:10:27.392598+0000","last_undegraded":"2023-11-28T11:10:27.392598+0000","last_fullsized":"2023-11-28T11:10:27.392598+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:35.327593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280664+0000","last_change":"2023-11-28T11:10:26.397086+0000","last_active":"2023-11-28T11:10:28.280664+0000","last_peered":"2023-11-28T11:10:28.280664+0000","last_clean":"2023-11-28T11:10:28.280664+0000","last_became_active":"2023-11-28T11:10:26.396819+0000","last_became_peered":"2023-11-28T11:10:26.396819+0000","last_unstale":"2023-11-28T11:10:28.280664+0000","last_undegraded":"2023-11-28T11:10:28.280664+0000","last_fullsized":"2023-11-28T11:10:28.280664+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:15.404234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280794+0000","last_change":"2023-11-28T11:10:25.996523+0000","last_active":"2023-11-28T11:10:28.280794+0000","last_peered":"2023-11-28T11:10:28.280794+0000","last_clean":"2023-11-28T11:10:28.280794+0000","last_became_active":"2023-11-28T11:10:25.996295+0000","last_became_peered":"2023-11-28T11:10:25.996295+0000","last_unstale":"2023-11-28T11:10:28.280794+0000","last_undegraded":"2023-11-28T11:10:28.280794+0000","last_fullsized":"2023-11-28T11:10:28.280794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:31.863853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280699+0000","last_change":"2023-11-28T11:10:26.403295+0000","last_active":"2023-11-28T11:10:28.280699+0000","last_peered":"2023-11-28T11:10:28.280699+0000","last_clean":"2023-11-28T11:10:28.280699+0000","last_became_active":"2023-11-28T11:10:26.403054+0000","last_became_peered":"2023-11-28T11:10:26.403054+0000","last_unstale":"2023-11-28T11:10:28.280699+0000","last_undegraded":"2023-11-28T11:10:28.280699+0000","last_fullsized":"2023-11-28T11:10:28.280699+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:58.033855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392985+0000","last_change":"2023-11-28T11:10:26.002400+0000","last_active":"2023-11-28T11:10:27.392985+0000","last_peered":"2023-11-28T11:10:27.392985+0000","last_clean":"2023-11-28T11:10:27.392985+0000","last_became_active":"2023-11-28T11:10:26.002005+0000","last_became_peered":"2023-11-28T11:10:26.002005+0000","last_unstale":"2023-11-28T11:10:27.392985+0000","last_undegraded":"2023-11-28T11:10:27.392985+0000","last_fullsized":"2023-11-28T11:10:27.392985+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:20.657127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392374+0000","last_change":"2023-11-28T11:10:26.409087+0000","last_active":"2023-11-28T11:10:27.392374+0000","last_peered":"2023-11-28T11:10:27.392374+0000","last_clean":"2023-11-28T11:10:27.392374+0000","last_became_active":"2023-11-28T11:10:26.408801+0000","last_became_peered":"2023-11-28T11:10:26.408801+0000","last_unstale":"2023-11-28T11:10:27.392374+0000","last_undegraded":"2023-11-28T11:10:27.392374+0000","last_fullsized":"2023-11-28T11:10:27.392374+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:24.728317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281297+0000","last_change":"2023-11-28T11:10:25.999334+0000","last_active":"2023-11-28T11:10:28.281297+0000","last_peered":"2023-11-28T11:10:28.281297+0000","last_clean":"2023-11-28T11:10:28.281297+0000","last_became_active":"2023-11-28T11:10:25.999059+0000","last_became_peered":"2023-11-28T11:10:25.999059+0000","last_unstale":"2023-11-28T11:10:28.281297+0000","last_undegraded":"2023-11-28T11:10:28.281297+0000","last_fullsized":"2023-11-28T11:10:28.281297+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:14.409235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392634+0000","last_change":"2023-11-28T11:10:26.413415+0000","last_active":"2023-11-28T11:10:27.392634+0000","last_peered":"2023-11-28T11:10:27.392634+0000","last_clean":"2023-11-28T11:10:27.392634+0000","last_became_active":"2023-11-28T11:10:26.410411+0000","last_became_peered":"2023-11-28T11:10:26.410411+0000","last_unstale":"2023-11-28T11:10:27.392634+0000","last_undegraded":"2023-11-28T11:10:27.392634+0000","last_fullsized":"2023-11-28T11:10:27.392634+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:47.464212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280476+0000","last_change":"2023-11-28T11:10:26.000932+0000","last_active":"2023-11-28T11:10:28.280476+0000","last_peered":"2023-11-28T11:10:28.280476+0000","last_clean":"2023-11-28T11:10:28.280476+0000","last_became_active":"2023-11-28T11:10:26.000693+0000","last_became_peered":"2023-11-28T11:10:26.000693+0000","last_unstale":"2023-11-28T11:10:28.280476+0000","last_undegraded":"2023-11-28T11:10:28.280476+0000","last_fullsized":"2023-11-28T11:10:28.280476+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:45.187578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282346+0000","last_change":"2023-11-28T11:10:26.411318+0000","last_active":"2023-11-28T11:10:28.282346+0000","last_peered":"2023-11-28T11:10:28.282346+0000","last_clean":"2023-11-28T11:10:28.282346+0000","last_became_active":"2023-11-28T11:10:26.411054+0000","last_became_peered":"2023-11-28T11:10:26.411054+0000","last_unstale":"2023-11-28T11:10:28.282346+0000","last_undegraded":"2023-11-28T11:10:28.282346+0000","last_fullsized":"2023-11-28T11:10:28.282346+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:17.210757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392644+0000","last_change":"2023-11-28T11:10:26.001327+0000","last_active":"2023-11-28T11:10:27.392644+0000","last_peered":"2023-11-28T11:10:27.392644+0000","last_clean":"2023-11-28T11:10:27.392644+0000","last_became_active":"2023-11-28T11:10:26.001002+0000","last_became_peered":"2023-11-28T11:10:26.001002+0000","last_unstale":"2023-11-28T11:10:27.392644+0000","last_undegraded":"2023-11-28T11:10:27.392644+0000","last_fullsized":"2023-11-28T11:10:27.392644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:21.675054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392502+0000","last_change":"2023-11-28T11:10:26.410059+0000","last_active":"2023-11-28T11:10:27.392502+0000","last_peered":"2023-11-28T11:10:27.392502+0000","last_clean":"2023-11-28T11:10:27.392502+0000","last_became_active":"2023-11-28T11:10:26.409692+0000","last_became_peered":"2023-11-28T11:10:26.409692+0000","last_unstale":"2023-11-28T11:10:27.392502+0000","last_undegraded":"2023-11-28T11:10:27.392502+0000","last_fullsized":"2023-11-28T11:10:27.392502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:03.682749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281116+0000","last_change":"2023-11-28T11:10:26.006026+0000","last_active":"2023-11-28T11:10:28.281116+0000","last_peered":"2023-11-28T11:10:28.281116+0000","last_clean":"2023-11-28T11:10:28.281116+0000","last_became_active":"2023-11-28T11:10:26.005788+0000","last_became_peered":"2023-11-28T11:10:26.005788+0000","last_unstale":"2023-11-28T11:10:28.281116+0000","last_undegraded":"2023-11-28T11:10:28.281116+0000","last_fullsized":"2023-11-28T11:10:28.281116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:21.915439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281967+0000","last_change":"2023-11-28T11:10:26.407533+0000","last_active":"2023-11-28T11:10:28.281967+0000","last_peered":"2023-11-28T11:10:28.281967+0000","last_clean":"2023-11-28T11:10:28.281967+0000","last_became_active":"2023-11-28T11:10:26.407224+0000","last_became_peered":"2023-11-28T11:10:26.407224+0000","last_unstale":"2023-11-28T11:10:28.281967+0000","last_undegraded":"2023-11-28T11:10:28.281967+0000","last_fullsized":"2023-11-28T11:10:28.281967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:18.125593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281720+0000","last_change":"2023-11-28T11:10:26.002832+0000","last_active":"2023-11-28T11:10:28.281720+0000","last_peered":"2023-11-28T11:10:28.281720+0000","last_clean":"2023-11-28T11:10:28.281720+0000","last_became_active":"2023-11-28T11:10:26.002538+0000","last_became_peered":"2023-11-28T11:10:26.002538+0000","last_unstale":"2023-11-28T11:10:28.281720+0000","last_undegraded":"2023-11-28T11:10:28.281720+0000","last_fullsized":"2023-11-28T11:10:28.281720+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:58:15.101170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281633+0000","last_change":"2023-11-28T11:10:26.413789+0000","last_active":"2023-11-28T11:10:28.281633+0000","last_peered":"2023-11-28T11:10:28.281633+0000","last_clean":"2023-11-28T11:10:28.281633+0000","last_became_active":"2023-11-28T11:10:26.412314+0000","last_became_peered":"2023-11-28T11:10:26.412314+0000","last_unstale":"2023-11-28T11:10:28.281633+0000","last_undegraded":"2023-11-28T11:10:28.281633+0000","last_fullsized":"2023-11-28T11:10:28.281633+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:17.299342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392651+0000","last_change":"2023-11-28T11:10:26.004872+0000","last_active":"2023-11-28T11:10:27.392651+0000","last_peered":"2023-11-28T11:10:27.392651+0000","last_clean":"2023-11-28T11:10:27.392651+0000","last_became_active":"2023-11-28T11:10:26.004566+0000","last_became_peered":"2023-11-28T11:10:26.004566+0000","last_unstale":"2023-11-28T11:10:27.392651+0000","last_undegraded":"2023-11-28T11:10:27.392651+0000","last_fullsized":"2023-11-28T11:10:27.392651+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:39.013445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491375+0000","last_change":"2023-11-28T11:10:26.402990+0000","last_active":"2023-11-28T11:10:27.491375+0000","last_peered":"2023-11-28T11:10:27.491375+0000","last_clean":"2023-11-28T11:10:27.491375+0000","last_became_active":"2023-11-28T11:10:26.402392+0000","last_became_peered":"2023-11-28T11:10:26.402392+0000","last_unstale":"2023-11-28T11:10:27.491375+0000","last_undegraded":"2023-11-28T11:10:27.491375+0000","last_fullsized":"2023-11-28T11:10:27.491375+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:40.706260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281970+0000","last_change":"2023-11-28T11:10:26.002113+0000","last_active":"2023-11-28T11:10:28.281970+0000","last_peered":"2023-11-28T11:10:28.281970+0000","last_clean":"2023-11-28T11:10:28.281970+0000","last_became_active":"2023-11-28T11:10:26.001825+0000","last_became_peered":"2023-11-28T11:10:26.001825+0000","last_unstale":"2023-11-28T11:10:28.281970+0000","last_undegraded":"2023-11-28T11:10:28.281970+0000","last_fullsized":"2023-11-28T11:10:28.281970+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:04.358989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281685+0000","last_change":"2023-11-28T11:10:26.414776+0000","last_active":"2023-11-28T11:10:28.281685+0000","last_peered":"2023-11-28T11:10:28.281685+0000","last_clean":"2023-11-28T11:10:28.281685+0000","last_became_active":"2023-11-28T11:10:26.414321+0000","last_became_peered":"2023-11-28T11:10:26.414321+0000","last_unstale":"2023-11-28T11:10:28.281685+0000","last_undegraded":"2023-11-28T11:10:28.281685+0000","last_fullsized":"2023-11-28T11:10:28.281685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:34.147447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281533+0000","last_change":"2023-11-28T11:10:25.997372+0000","last_active":"2023-11-28T11:10:28.281533+0000","last_peered":"2023-11-28T11:10:28.281533+0000","last_clean":"2023-11-28T11:10:28.281533+0000","last_became_active":"2023-11-28T11:10:25.997088+0000","last_became_peered":"2023-11-28T11:10:25.997088+0000","last_unstale":"2023-11-28T11:10:28.281533+0000","last_undegraded":"2023-11-28T11:10:28.281533+0000","last_fullsized":"2023-11-28T11:10:28.281533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:35:05.239030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392607+0000","last_change":"2023-11-28T11:10:26.406113+0000","last_active":"2023-11-28T11:10:27.392607+0000","last_peered":"2023-11-28T11:10:27.392607+0000","last_clean":"2023-11-28T11:10:27.392607+0000","last_became_active":"2023-11-28T11:10:26.405825+0000","last_became_peered":"2023-11-28T11:10:26.405825+0000","last_unstale":"2023-11-28T11:10:27.392607+0000","last_undegraded":"2023-11-28T11:10:27.392607+0000","last_fullsized":"2023-11-28T11:10:27.392607+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:51.366529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392762+0000","last_change":"2023-11-28T11:10:26.004985+0000","last_active":"2023-11-28T11:10:27.392762+0000","last_peered":"2023-11-28T11:10:27.392762+0000","last_clean":"2023-11-28T11:10:27.392762+0000","last_became_active":"2023-11-28T11:10:26.004724+0000","last_became_peered":"2023-11-28T11:10:26.004724+0000","last_unstale":"2023-11-28T11:10:27.392762+0000","last_undegraded":"2023-11-28T11:10:27.392762+0000","last_fullsized":"2023-11-28T11:10:27.392762+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:18.695344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281326+0000","last_change":"2023-11-28T11:10:26.410716+0000","last_active":"2023-11-28T11:10:28.281326+0000","last_peered":"2023-11-28T11:10:28.281326+0000","last_clean":"2023-11-28T11:10:28.281326+0000","last_became_active":"2023-11-28T11:10:26.410434+0000","last_became_peered":"2023-11-28T11:10:26.410434+0000","last_unstale":"2023-11-28T11:10:28.281326+0000","last_undegraded":"2023-11-28T11:10:28.281326+0000","last_fullsized":"2023-11-28T11:10:28.281326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:15.419778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282211+0000","last_change":"2023-11-28T11:10:25.998724+0000","last_active":"2023-11-28T11:10:28.282211+0000","last_peered":"2023-11-28T11:10:28.282211+0000","last_clean":"2023-11-28T11:10:28.282211+0000","last_became_active":"2023-11-28T11:10:25.998357+0000","last_became_peered":"2023-11-28T11:10:25.998357+0000","last_unstale":"2023-11-28T11:10:28.282211+0000","last_undegraded":"2023-11-28T11:10:28.282211+0000","last_fullsized":"2023-11-28T11:10:28.282211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:37.166206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281016+0000","last_change":"2023-11-28T11:10:26.413177+0000","last_active":"2023-11-28T11:10:28.281016+0000","last_peered":"2023-11-28T11:10:28.281016+0000","last_clean":"2023-11-28T11:10:28.281016+0000","last_became_active":"2023-11-28T11:10:26.412942+0000","last_became_peered":"2023-11-28T11:10:26.412942+0000","last_unstale":"2023-11-28T11:10:28.281016+0000","last_undegraded":"2023-11-28T11:10:28.281016+0000","last_fullsized":"2023-11-28T11:10:28.281016+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:35.566533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392622+0000","last_change":"2023-11-28T11:10:26.000242+0000","last_active":"2023-11-28T11:10:27.392622+0000","last_peered":"2023-11-28T11:10:27.392622+0000","last_clean":"2023-11-28T11:10:27.392622+0000","last_became_active":"2023-11-28T11:10:25.999986+0000","last_became_peered":"2023-11-28T11:10:25.999986+0000","last_unstale":"2023-11-28T11:10:27.392622+0000","last_undegraded":"2023-11-28T11:10:27.392622+0000","last_fullsized":"2023-11-28T11:10:27.392622+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:56.105652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281225+0000","last_change":"2023-11-28T11:10:26.414086+0000","last_active":"2023-11-28T11:10:28.281225+0000","last_peered":"2023-11-28T11:10:28.281225+0000","last_clean":"2023-11-28T11:10:28.281225+0000","last_became_active":"2023-11-28T11:10:26.412296+0000","last_became_peered":"2023-11-28T11:10:26.412296+0000","last_unstale":"2023-11-28T11:10:28.281225+0000","last_undegraded":"2023-11-28T11:10:28.281225+0000","last_fullsized":"2023-11-28T11:10:28.281225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:54.076510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280962+0000","last_change":"2023-11-28T11:10:26.003726+0000","last_active":"2023-11-28T11:10:28.280962+0000","last_peered":"2023-11-28T11:10:28.280962+0000","last_clean":"2023-11-28T11:10:28.280962+0000","last_became_active":"2023-11-28T11:10:26.003443+0000","last_became_peered":"2023-11-28T11:10:26.003443+0000","last_unstale":"2023-11-28T11:10:28.280962+0000","last_undegraded":"2023-11-28T11:10:28.280962+0000","last_fullsized":"2023-11-28T11:10:28.280962+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:20.503700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393088+0000","last_change":"2023-11-28T11:10:26.401038+0000","last_active":"2023-11-28T11:10:27.393088+0000","last_peered":"2023-11-28T11:10:27.393088+0000","last_clean":"2023-11-28T11:10:27.393088+0000","last_became_active":"2023-11-28T11:10:26.400755+0000","last_became_peered":"2023-11-28T11:10:26.400755+0000","last_unstale":"2023-11-28T11:10:27.393088+0000","last_undegraded":"2023-11-28T11:10:27.393088+0000","last_fullsized":"2023-11-28T11:10:27.393088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:43.887933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491613+0000","last_change":"2023-11-28T11:10:25.999626+0000","last_active":"2023-11-28T11:10:27.491613+0000","last_peered":"2023-11-28T11:10:27.491613+0000","last_clean":"2023-11-28T11:10:27.491613+0000","last_became_active":"2023-11-28T11:10:25.999348+0000","last_became_peered":"2023-11-28T11:10:25.999348+0000","last_unstale":"2023-11-28T11:10:27.491613+0000","last_undegraded":"2023-11-28T11:10:27.491613+0000","last_fullsized":"2023-11-28T11:10:27.491613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:43.906653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393175+0000","last_change":"2023-11-28T11:10:26.402805+0000","last_active":"2023-11-28T11:10:27.393175+0000","last_peered":"2023-11-28T11:10:27.393175+0000","last_clean":"2023-11-28T11:10:27.393175+0000","last_became_active":"2023-11-28T11:10:26.402547+0000","last_became_peered":"2023-11-28T11:10:26.402547+0000","last_unstale":"2023-11-28T11:10:27.393175+0000","last_undegraded":"2023-11-28T11:10:27.393175+0000","last_fullsized":"2023-11-28T11:10:27.393175+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:38.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280877+0000","last_change":"2023-11-28T11:10:25.998272+0000","last_active":"2023-11-28T11:10:28.280877+0000","last_peered":"2023-11-28T11:10:28.280877+0000","last_clean":"2023-11-28T11:10:28.280877+0000","last_became_active":"2023-11-28T11:10:25.998018+0000","last_became_peered":"2023-11-28T11:10:25.998018+0000","last_unstale":"2023-11-28T11:10:28.280877+0000","last_undegraded":"2023-11-28T11:10:28.280877+0000","last_fullsized":"2023-11-28T11:10:28.280877+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:39.013445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491482+0000","last_change":"2023-11-28T11:10:26.403151+0000","last_active":"2023-11-28T11:10:27.491482+0000","last_peered":"2023-11-28T11:10:27.491482+0000","last_clean":"2023-11-28T11:10:27.491482+0000","last_became_active":"2023-11-28T11:10:26.402727+0000","last_became_peered":"2023-11-28T11:10:26.402727+0000","last_unstale":"2023-11-28T11:10:27.491482+0000","last_undegraded":"2023-11-28T11:10:27.491482+0000","last_fullsized":"2023-11-28T11:10:27.491482+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:57.116195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281826+0000","last_change":"2023-11-28T11:10:26.000020+0000","last_active":"2023-11-28T11:10:28.281826+0000","last_peered":"2023-11-28T11:10:28.281826+0000","last_clean":"2023-11-28T11:10:28.281826+0000","last_became_active":"2023-11-28T11:10:25.999724+0000","last_became_peered":"2023-11-28T11:10:25.999724+0000","last_unstale":"2023-11-28T11:10:28.281826+0000","last_undegraded":"2023-11-28T11:10:28.281826+0000","last_fullsized":"2023-11-28T11:10:28.281826+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:15.220689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280764+0000","last_change":"2023-11-28T11:10:26.403140+0000","last_active":"2023-11-28T11:10:28.280764+0000","last_peered":"2023-11-28T11:10:28.280764+0000","last_clean":"2023-11-28T11:10:28.280764+0000","last_became_active":"2023-11-28T11:10:26.402678+0000","last_became_peered":"2023-11-28T11:10:26.402678+0000","last_unstale":"2023-11-28T11:10:28.280764+0000","last_undegraded":"2023-11-28T11:10:28.280764+0000","last_fullsized":"2023-11-28T11:10:28.280764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:18.440998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393094+0000","last_change":"2023-11-28T11:10:25.999027+0000","last_active":"2023-11-28T11:10:27.393094+0000","last_peered":"2023-11-28T11:10:27.393094+0000","last_clean":"2023-11-28T11:10:27.393094+0000","last_became_active":"2023-11-28T11:10:25.998765+0000","last_became_peered":"2023-11-28T11:10:25.998765+0000","last_unstale":"2023-11-28T11:10:27.393094+0000","last_undegraded":"2023-11-28T11:10:27.393094+0000","last_fullsized":"2023-11-28T11:10:27.393094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:54.368325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392427+0000","last_change":"2023-11-28T11:10:26.414745+0000","last_active":"2023-11-28T11:10:27.392427+0000","last_peered":"2023-11-28T11:10:27.392427+0000","last_clean":"2023-11-28T11:10:27.392427+0000","last_became_active":"2023-11-28T11:10:26.413422+0000","last_became_peered":"2023-11-28T11:10:26.413422+0000","last_unstale":"2023-11-28T11:10:27.392427+0000","last_undegraded":"2023-11-28T11:10:27.392427+0000","last_fullsized":"2023-11-28T11:10:27.392427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:50.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491328+0000","last_change":"2023-11-28T11:10:25.993542+0000","last_active":"2023-11-28T11:10:27.491328+0000","last_peered":"2023-11-28T11:10:27.491328+0000","last_clean":"2023-11-28T11:10:27.491328+0000","last_became_active":"2023-11-28T11:10:25.993231+0000","last_became_peered":"2023-11-28T11:10:25.993231+0000","last_unstale":"2023-11-28T11:10:27.491328+0000","last_undegraded":"2023-11-28T11:10:27.491328+0000","last_fullsized":"2023-11-28T11:10:27.491328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:18:06.469023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281434+0000","last_change":"2023-11-28T11:10:26.413797+0000","last_active":"2023-11-28T11:10:28.281434+0000","last_peered":"2023-11-28T11:10:28.281434+0000","last_clean":"2023-11-28T11:10:28.281434+0000","last_became_active":"2023-11-28T11:10:26.413465+0000","last_became_peered":"2023-11-28T11:10:26.413465+0000","last_unstale":"2023-11-28T11:10:28.281434+0000","last_undegraded":"2023-11-28T11:10:28.281434+0000","last_fullsized":"2023-11-28T11:10:28.281434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:18.125593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281654+0000","last_change":"2023-11-28T11:10:25.997395+0000","last_active":"2023-11-28T11:10:28.281654+0000","last_peered":"2023-11-28T11:10:28.281654+0000","last_clean":"2023-11-28T11:10:28.281654+0000","last_became_active":"2023-11-28T11:10:25.997075+0000","last_became_peered":"2023-11-28T11:10:25.997075+0000","last_unstale":"2023-11-28T11:10:28.281654+0000","last_undegraded":"2023-11-28T11:10:28.281654+0000","last_fullsized":"2023-11-28T11:10:28.281654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:42.877390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491243+0000","last_change":"2023-11-28T11:10:26.407601+0000","last_active":"2023-11-28T11:10:27.491243+0000","last_peered":"2023-11-28T11:10:27.491243+0000","last_clean":"2023-11-28T11:10:27.491243+0000","last_became_active":"2023-11-28T11:10:26.407326+0000","last_became_peered":"2023-11-28T11:10:26.407326+0000","last_unstale":"2023-11-28T11:10:27.491243+0000","last_undegraded":"2023-11-28T11:10:27.491243+0000","last_fullsized":"2023-11-28T11:10:27.491243+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:21.667670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491311+0000","last_change":"2023-11-28T11:10:25.994970+0000","last_active":"2023-11-28T11:10:27.491311+0000","last_peered":"2023-11-28T11:10:27.491311+0000","last_clean":"2023-11-28T11:10:27.491311+0000","last_became_active":"2023-11-28T11:10:25.994492+0000","last_became_peered":"2023-11-28T11:10:25.994492+0000","last_unstale":"2023-11-28T11:10:27.491311+0000","last_undegraded":"2023-11-28T11:10:27.491311+0000","last_fullsized":"2023-11-28T11:10:27.491311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:17.700264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392522+0000","last_change":"2023-11-28T11:10:26.410677+0000","last_active":"2023-11-28T11:10:27.392522+0000","last_peered":"2023-11-28T11:10:27.392522+0000","last_clean":"2023-11-28T11:10:27.392522+0000","last_became_active":"2023-11-28T11:10:26.409353+0000","last_became_peered":"2023-11-28T11:10:26.409353+0000","last_unstale":"2023-11-28T11:10:27.392522+0000","last_undegraded":"2023-11-28T11:10:27.392522+0000","last_fullsized":"2023-11-28T11:10:27.392522+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:27:38.780586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392372+0000","last_change":"2023-11-28T11:10:25.997169+0000","last_active":"2023-11-28T11:10:27.392372+0000","last_peered":"2023-11-28T11:10:27.392372+0000","last_clean":"2023-11-28T11:10:27.392372+0000","last_became_active":"2023-11-28T11:10:25.996863+0000","last_became_peered":"2023-11-28T11:10:25.996863+0000","last_unstale":"2023-11-28T11:10:27.392372+0000","last_undegraded":"2023-11-28T11:10:27.392372+0000","last_fullsized":"2023-11-28T11:10:27.392372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:31:34.555990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281791+0000","last_change":"2023-11-28T11:10:26.414606+0000","last_active":"2023-11-28T11:10:28.281791+0000","last_peered":"2023-11-28T11:10:28.281791+0000","last_clean":"2023-11-28T11:10:28.281791+0000","last_became_active":"2023-11-28T11:10:26.413966+0000","last_became_peered":"2023-11-28T11:10:26.413966+0000","last_unstale":"2023-11-28T11:10:28.281791+0000","last_undegraded":"2023-11-28T11:10:28.281791+0000","last_fullsized":"2023-11-28T11:10:28.281791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:22.925982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280727+0000","last_change":"2023-11-28T11:10:26.000429+0000","last_active":"2023-11-28T11:10:28.280727+0000","last_peered":"2023-11-28T11:10:28.280727+0000","last_clean":"2023-11-28T11:10:28.280727+0000","last_became_active":"2023-11-28T11:10:26.000087+0000","last_became_peered":"2023-11-28T11:10:26.000087+0000","last_unstale":"2023-11-28T11:10:28.280727+0000","last_undegraded":"2023-11-28T11:10:28.280727+0000","last_fullsized":"2023-11-28T11:10:28.280727+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:25.875992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.400777+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:10:26.400777+0000","last_peered":"2023-11-28T11:10:26.400777+0000","last_clean":"2023-11-28T11:10:26.400777+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:10:26.400777+0000","last_undegraded":"2023-11-28T11:10:26.400777+0000","last_fullsized":"2023-11-28T11:10:26.400777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392629+0000","last_change":"2023-11-28T11:10:26.410088+0000","last_active":"2023-11-28T11:10:27.392629+0000","last_peered":"2023-11-28T11:10:27.392629+0000","last_clean":"2023-11-28T11:10:27.392629+0000","last_became_active":"2023-11-28T11:10:26.409822+0000","last_became_peered":"2023-11-28T11:10:26.409822+0000","last_unstale":"2023-11-28T11:10:27.392629+0000","last_undegraded":"2023-11-28T11:10:27.392629+0000","last_fullsized":"2023-11-28T11:10:27.392629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:25.040675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393067+0000","last_change":"2023-11-28T11:10:25.999971+0000","last_active":"2023-11-28T11:10:27.393067+0000","last_peered":"2023-11-28T11:10:27.393067+0000","last_clean":"2023-11-28T11:10:27.393067+0000","last_became_active":"2023-11-28T11:10:25.999647+0000","last_became_peered":"2023-11-28T11:10:25.999647+0000","last_unstale":"2023-11-28T11:10:27.393067+0000","last_undegraded":"2023-11-28T11:10:27.393067+0000","last_fullsized":"2023-11-28T11:10:27.393067+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:06.628905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.398000+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:10:26.398000+0000","last_peered":"2023-11-28T11:10:26.398000+0000","last_clean":"2023-11-28T11:10:26.398000+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:10:26.398000+0000","last_undegraded":"2023-11-28T11:10:26.398000+0000","last_fullsized":"2023-11-28T11:10:26.398000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280638+0000","last_change":"2023-11-28T11:10:26.408832+0000","last_active":"2023-11-28T11:10:28.280638+0000","last_peered":"2023-11-28T11:10:28.280638+0000","last_clean":"2023-11-28T11:10:28.280638+0000","last_became_active":"2023-11-28T11:10:26.408448+0000","last_became_peered":"2023-11-28T11:10:26.408448+0000","last_unstale":"2023-11-28T11:10:28.280638+0000","last_undegraded":"2023-11-28T11:10:28.280638+0000","last_fullsized":"2023-11-28T11:10:28.280638+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:58.718873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.491375+0000","last_change":"2023-11-28T11:10:25.992979+0000","last_active":"2023-11-28T11:10:27.491375+0000","last_peered":"2023-11-28T11:10:27.491375+0000","last_clean":"2023-11-28T11:10:27.491375+0000","last_became_active":"2023-11-28T11:10:25.992711+0000","last_became_peered":"2023-11-28T11:10:25.992711+0000","last_unstale":"2023-11-28T11:10:27.491375+0000","last_undegraded":"2023-11-28T11:10:27.491375+0000","last_fullsized":"2023-11-28T11:10:27.491375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:37.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.388424+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:10:26.388424+0000","last_peered":"2023-11-28T11:10:26.388424+0000","last_clean":"2023-11-28T11:10:26.388424+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:10:26.388424+0000","last_undegraded":"2023-11-28T11:10:26.388424+0000","last_fullsized":"2023-11-28T11:10:26.388424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":187,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.390844+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:10:26.390844+0000","last_peered":"2023-11-28T11:10:26.390844+0000","last_clean":"2023-11-28T11:10:26.390844+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:10:26.390844+0000","last_undegraded":"2023-11-28T11:10:26.390844+0000","last_fullsized":"2023-11-28T11:10:26.390844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392760+0000","last_change":"2023-11-28T11:10:26.402761+0000","last_active":"2023-11-28T11:10:27.392760+0000","last_peered":"2023-11-28T11:10:27.392760+0000","last_clean":"2023-11-28T11:10:27.392760+0000","last_became_active":"2023-11-28T11:10:26.402473+0000","last_became_peered":"2023-11-28T11:10:26.402473+0000","last_unstale":"2023-11-28T11:10:27.392760+0000","last_undegraded":"2023-11-28T11:10:27.392760+0000","last_fullsized":"2023-11-28T11:10:27.392760+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:35.131013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392807+0000","last_change":"2023-11-28T11:10:26.002802+0000","last_active":"2023-11-28T11:10:27.392807+0000","last_peered":"2023-11-28T11:10:27.392807+0000","last_clean":"2023-11-28T11:10:27.392807+0000","last_became_active":"2023-11-28T11:10:26.002540+0000","last_became_peered":"2023-11-28T11:10:26.002540+0000","last_unstale":"2023-11-28T11:10:27.392807+0000","last_undegraded":"2023-11-28T11:10:27.392807+0000","last_fullsized":"2023-11-28T11:10:27.392807+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:58.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.391484+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:10:26.391484+0000","last_peered":"2023-11-28T11:10:26.391484+0000","last_clean":"2023-11-28T11:10:26.391484+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:10:26.391484+0000","last_undegraded":"2023-11-28T11:10:26.391484+0000","last_fullsized":"2023-11-28T11:10:26.391484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392366+0000","last_change":"2023-11-28T11:10:26.408598+0000","last_active":"2023-11-28T11:10:27.392366+0000","last_peered":"2023-11-28T11:10:27.392366+0000","last_clean":"2023-11-28T11:10:27.392366+0000","last_became_active":"2023-11-28T11:10:26.408300+0000","last_became_peered":"2023-11-28T11:10:26.408300+0000","last_unstale":"2023-11-28T11:10:27.392366+0000","last_undegraded":"2023-11-28T11:10:27.392366+0000","last_fullsized":"2023-11-28T11:10:27.392366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:56.623506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.282104+0000","last_change":"2023-11-28T11:10:26.000392+0000","last_active":"2023-11-28T11:10:28.282104+0000","last_peered":"2023-11-28T11:10:28.282104+0000","last_clean":"2023-11-28T11:10:28.282104+0000","last_became_active":"2023-11-28T11:10:26.000110+0000","last_became_peered":"2023-11-28T11:10:26.000110+0000","last_unstale":"2023-11-28T11:10:28.282104+0000","last_undegraded":"2023-11-28T11:10:28.282104+0000","last_fullsized":"2023-11-28T11:10:28.282104+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:57.123247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.390737+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:10:26.390737+0000","last_peered":"2023-11-28T11:10:26.390737+0000","last_clean":"2023-11-28T11:10:26.390737+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:10:26.390737+0000","last_undegraded":"2023-11-28T11:10:26.390737+0000","last_fullsized":"2023-11-28T11:10:26.390737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280447+0000","last_change":"2023-11-28T11:10:26.404780+0000","last_active":"2023-11-28T11:10:28.280447+0000","last_peered":"2023-11-28T11:10:28.280447+0000","last_clean":"2023-11-28T11:10:28.280447+0000","last_became_active":"2023-11-28T11:10:26.404521+0000","last_became_peered":"2023-11-28T11:10:26.404521+0000","last_unstale":"2023-11-28T11:10:28.280447+0000","last_undegraded":"2023-11-28T11:10:28.280447+0000","last_fullsized":"2023-11-28T11:10:28.280447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:50.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392560+0000","last_change":"2023-11-28T11:10:26.002760+0000","last_active":"2023-11-28T11:10:27.392560+0000","last_peered":"2023-11-28T11:10:27.392560+0000","last_clean":"2023-11-28T11:10:27.392560+0000","last_became_active":"2023-11-28T11:10:26.002505+0000","last_became_peered":"2023-11-28T11:10:26.002505+0000","last_unstale":"2023-11-28T11:10:27.392560+0000","last_undegraded":"2023-11-28T11:10:27.392560+0000","last_fullsized":"2023-11-28T11:10:27.392560+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:54.816419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.393214+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:10:26.393214+0000","last_peered":"2023-11-28T11:10:26.393214+0000","last_clean":"2023-11-28T11:10:26.393214+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:10:26.393214+0000","last_undegraded":"2023-11-28T11:10:26.393214+0000","last_fullsized":"2023-11-28T11:10:26.393214+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.281887+0000","last_change":"2023-11-28T11:10:26.409836+0000","last_active":"2023-11-28T11:10:28.281887+0000","last_peered":"2023-11-28T11:10:28.281887+0000","last_clean":"2023-11-28T11:10:28.281887+0000","last_became_active":"2023-11-28T11:10:26.405152+0000","last_became_peered":"2023-11-28T11:10:26.405152+0000","last_unstale":"2023-11-28T11:10:28.281887+0000","last_undegraded":"2023-11-28T11:10:28.281887+0000","last_fullsized":"2023-11-28T11:10:28.281887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:51.366529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.392662+0000","last_change":"2023-11-28T11:10:26.005110+0000","last_active":"2023-11-28T11:10:27.392662+0000","last_peered":"2023-11-28T11:10:27.392662+0000","last_clean":"2023-11-28T11:10:27.392662+0000","last_became_active":"2023-11-28T11:10:26.004815+0000","last_became_peered":"2023-11-28T11:10:26.004815+0000","last_unstale":"2023-11-28T11:10:27.392662+0000","last_undegraded":"2023-11-28T11:10:27.392662+0000","last_fullsized":"2023-11-28T11:10:27.392662+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:45.187578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.396773+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:10:26.396773+0000","last_peered":"2023-11-28T11:10:26.396773+0000","last_clean":"2023-11-28T11:10:26.396773+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:10:26.396773+0000","last_undegraded":"2023-11-28T11:10:26.396773+0000","last_fullsized":"2023-11-28T11:10:26.396773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:27.393105+0000","last_change":"2023-11-28T11:10:26.406365+0000","last_active":"2023-11-28T11:10:27.393105+0000","last_peered":"2023-11-28T11:10:27.393105+0000","last_clean":"2023-11-28T11:10:27.393105+0000","last_became_active":"2023-11-28T11:10:26.406065+0000","last_became_peered":"2023-11-28T11:10:26.406065+0000","last_unstale":"2023-11-28T11:10:27.393105+0000","last_undegraded":"2023-11-28T11:10:27.393105+0000","last_fullsized":"2023-11-28T11:10:27.393105+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:25.971699+0000","last_clean_scrub_stamp":"2023-11-28T11:10:25.971699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:34.147447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T11:10:28.280807+0000","last_change":"2023-11-28T11:10:25.994288+0000","last_active":"2023-11-28T11:10:28.280807+0000","last_peered":"2023-11-28T11:10:28.280807+0000","last_clean":"2023-11-28T11:10:28.280807+0000","last_became_active":"2023-11-28T11:10:25.994000+0000","last_became_peered":"2023-11-28T11:10:25.994000+0000","last_unstale":"2023-11-28T11:10:28.280807+0000","last_undegraded":"2023-11-28T11:10:28.280807+0000","last_fullsized":"2023-11-28T11:10:28.280807+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:24.961156+0000","last_clean_scrub_stamp":"2023-11-28T11:10:24.961156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:17.115051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T11:10:26.399592+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:10:26.399592+0000","last_peered":"2023-11-28T11:10:26.399592+0000","last_clean":"2023-11-28T11:10:26.399592+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:10:26.399592+0000","last_undegraded":"2023-11-28T11:10:26.399592+0000","last_fullsized":"2023-11-28T11:10:26.399592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10276,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361564,"statfs":{"total":96636764160,"available":96626241536,"internally_reserved":0,"allocated":1208320,"data_stored":870716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.57999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39400000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.501}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.621}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9032,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362808,"statfs":{"total":96636764160,"available":96627515392,"internally_reserved":0,"allocated":589824,"data_stored":271854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.629},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.67800000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.65000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.504}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.61499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10208,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361632,"statfs":{"total":96636764160,"available":96626311168,"internally_reserved":0,"allocated":1204224,"data_stored":870646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9240,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362600,"statfs":{"total":96636764160,"available":96627302400,"internally_reserved":0,"allocated":606208,"data_stored":273968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.27000000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.34300000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.6429999999999998,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.73399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.50800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9056,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362784,"statfs":{"total":96636764160,"available":96627490816,"internally_reserved":0,"allocated":614400,"data_stored":280244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.309}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.50900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9248,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362592,"statfs":{"total":96636764160,"available":96627294208,"internally_reserved":0,"allocated":614400,"data_stored":280244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.133,"5min":5.133,"15min":5.133},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9252,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362588,"statfs":{"total":96636764160,"available":96627290112,"internally_reserved":0,"allocated":618496,"data_stored":281736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.64700000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57099999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.378},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.44900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42999999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.505}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9120,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362720,"statfs":{"total":96636764160,"available":96627425280,"internally_reserved":0,"allocated":614400,"data_stored":280824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":8.4610000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":8.4830000000000005,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.601,"5min":6.601,"15min":6.601},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.627},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.45700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:30.829 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:10:30.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:10:30.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.990+0000 7f4fc401b700 1 Processor -- start 2023-11-28T11:10:30.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.990+0000 7f4fc401b700 1 -- start start 2023-11-28T11:10:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc401b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0ca3a0 0x7f4fbc0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc401b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc0ca810 con 0x7f4fbc0cbfc0 2023-11-28T11:10:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc401b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc0caaf0 con 0x7f4fbc0cd530 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc401b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cadd0 con 0x7f4fbc0ca3a0 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc25b8700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4fbc0cbfc0 0x7f4fbc15d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49602/0 (socket says 172.21.15.83:49602) 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc25b8700 1 -- 172.21.15.83:0/1099659029 learned_addr learned my addr 172.21.15.83:0/1099659029 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc15b6700 1 --2- 172.21.15.83:0/1099659029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0ca3a0 0x7f4fbc0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856324315 0 0) 0x7f4fbc0ca810 con 0x7f4fbc0cbfc0 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa80035b0 con 0x7f4fbc0cbfc0 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833587521 0 0) 0x7f4fbc0caaf0 con 0x7f4fbc0cd530 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fbc0ca810 con 0x7f4fbc0cd530 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc15b6700 1 -- 172.21.15.83:0/1099659029 >> v1:172.21.15.90:6789/0 conn(0x7f4fbc0cd530 legacy=0x7f4fbc0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660640040 0 0) 0x7f4fa80035b0 con 0x7f4fbc0cbfc0 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.991+0000 7f4fc15b6700 1 -- 172.21.15.83:0/1099659029 >> v1:172.21.15.83:6789/0 conn(0x7f4fbc0cbfc0 legacy=0x7f4fbc15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc15b6700 1 -- 172.21.15.83:0/1099659029 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0cb0b0 con 0x7f4fbc0ca3a0 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc15b6700 1 --2- 172.21.15.83:0/1099659029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0ca3a0 0x7f4fbc0b05a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009870 tx=0x7f4fb8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541c1cfdb2f7b84c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40019f0 con 0x7f4fbc0ca3a0 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb8004030 con 0x7f4fbc0ca3a0 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc0db5700 1 -- 172.21.15.83:0/1099659029 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80043c0 con 0x7f4fbc0ca3a0 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc401b700 1 -- 172.21.15.83:0/1099659029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0ca3a0 msgr2=0x7f4fbc0b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc401b700 1 --2- 172.21.15.83:0/1099659029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0ca3a0 0x7f4fbc0b05a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009870 tx=0x7f4fb8009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc401b700 1 -- 172.21.15.83:0/1099659029 shutdown_connections 2023-11-28T11:10:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc401b700 1 --2- 172.21.15.83:0/1099659029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0ca3a0 0x7f4fbc0b05a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.992+0000 7f4fc401b700 1 -- 172.21.15.83:0/1099659029 >> 172.21.15.83:0/1099659029 conn(0x7f4fbc0b2ca0 msgr2=0x7f4fbc0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.993+0000 7f4fc401b700 1 -- 172.21.15.83:0/1099659029 shutdown_connections 2023-11-28T11:10:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.993+0000 7f4fc401b700 1 -- 172.21.15.83:0/1099659029 wait complete. 2023-11-28T11:10:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 Processor -- start 2023-11-28T11:10:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 -- start start 2023-11-28T11:10:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fbc0ca3a0 0x7f4fbc14f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 0x7f4fbc0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0cd530 0x7f4fbc0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c4ec0 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c51a0 con 0x7f4fbc0ca3a0 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.994+0000 7f4fc401b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c5480 con 0x7f4fbc0cd530 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 0x7f4fbc0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 0x7f4fbc0ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51746/0 (socket says 172.21.15.83:51746) 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 -- 172.21.15.83:0/240033885 learned_addr learned my addr 172.21.15.83:0/240033885 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc25b8700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0cd530 0x7f4fbc0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc1db7700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fbc0ca3a0 0x7f4fbc14f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 -- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0cd530 msgr2=0x7f4fbc0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0cd530 0x7f4fbc0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 -- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fbc0ca3a0 msgr2=0x7f4fbc14f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fbc0ca3a0 0x7f4fbc14f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 -- 172.21.15.83:0/240033885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb80094a0 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc1db7700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fbc0ca3a0 0x7f4fbc14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc15b6700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 0x7f4fbc0ad870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8011630 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb801f070 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80119f0 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc0b3820 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.995+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc0cb390 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.996+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc0b39b0 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.996+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4fb8017020 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.997+0000 7f4fb27fc700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4fa00682f0 0x7f4fa006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.997+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4fb8021070 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.997+0000 7f4fc1db7700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4fa00682f0 0x7f4fa006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:30.997+0000 7f4fc1db7700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4fa00682f0 0x7f4fa006a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0011e0 tx=0x7f4fac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.001+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4fb8053e40 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.259+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f4fbc074240 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.263+0000 7f4fb27fc700 1 -- 172.21.15.83:0/240033885 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4fb8057460 con 0x7f4fbc0cbfc0 2023-11-28T11:10:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4fa00682f0 msgr2=0x7f4fa006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4fa00682f0 0x7f4fa006a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0011e0 tx=0x7f4fac009040 comp rx=0 tx=0).stop 2023-11-28T11:10:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 msgr2=0x7f4fbc0ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 0x7f4fbc0ad870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb8004750 comp rx=0 tx=0).stop 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 shutdown_connections 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4fbc0cd530 0x7f4fbc0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fbc0ca3a0 0x7f4fbc14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4fa00682f0 0x7f4fa006a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 --2- 172.21.15.83:0/240033885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fbc0cbfc0 0x7f4fbc0ad870 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 >> 172.21.15.83:0/240033885 conn(0x7f4fbc0b2ca0 msgr2=0x7f4fbc0cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 shutdown_connections 2023-11-28T11:10:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.265+0000 7f4fc401b700 1 -- 172.21.15.83:0/240033885 wait complete. 2023-11-28T11:10:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-28T11:10:31.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:31.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.396+0000 7f75b72d4700 1 Processor -- start 2023-11-28T11:10:31.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.396+0000 7f75b72d4700 1 -- start start 2023-11-28T11:10:31.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b72d4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b015a850 0x7f75b00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b72d4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75b00b4270 0x7f75b00b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b72d4700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b015efa0 con 0x7f75b015a480 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b72d4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b00ceab0 con 0x7f75b00b4270 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b72d4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00cecc0 con 0x7f75b015a850 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75b00b4270 0x7f75b00b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75b00b4270 0x7f75b00b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51756/0 (socket says 172.21.15.83:51756) 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 -- 172.21.15.83:0/2831385667 learned_addr learned my addr 172.21.15.83:0/2831385667 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b486f700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b015a850 0x7f75b00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 -- 172.21.15.83:0/2831385667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b015a850 msgr2=0x7f75b00b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b015a850 0x7f75b00b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 -- 172.21.15.83:0/2831385667 >> v1:172.21.15.90:6789/0 conn(0x7f75b015a480 legacy=0x7f75b00cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 -- 172.21.15.83:0/2831385667 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00cefa0 con 0x7f75b00b4270 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b486f700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b015a850 0x7f75b00b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75b5871700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b00b4270 0x7f75b00b2960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009e50 tx=0x7f75a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271107ba01e11523 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75a7fff700 1 -- 172.21.15.83:0/2831385667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a800e070 con 0x7f75b00b4270 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75a7fff700 1 -- 172.21.15.83:0/2831385667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8003a20 con 0x7f75b00b4270 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.397+0000 7f75a7fff700 1 -- 172.21.15.83:0/2831385667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003dd0 con 0x7f75b00b4270 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 -- 172.21.15.83:0/2831385667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b00b4270 msgr2=0x7f75b00b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b00b4270 0x7f75b00b2960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009e50 tx=0x7f75a800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 -- 172.21.15.83:0/2831385667 shutdown_connections 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b00b4270 0x7f75b00b2960 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 --2- 172.21.15.83:0/2831385667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b015a850 0x7f75b00b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 -- 172.21.15.83:0/2831385667 >> 172.21.15.83:0/2831385667 conn(0x7f75b006de20 msgr2=0x7f75b006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 -- 172.21.15.83:0/2831385667 shutdown_connections 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.398+0000 7f75b72d4700 1 -- 172.21.15.83:0/2831385667 wait complete. 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.399+0000 7f75b72d4700 1 Processor -- start 2023-11-28T11:10:31.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.399+0000 7f75b72d4700 1 -- start start 2023-11-28T11:10:31.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b72d4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b00b4270 0x7f75b0153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b72d4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 0x7f75b0154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b72d4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b015a850 0x7f75b01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b72d4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01f0010 con 0x7f75b015a480 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b72d4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01560a0 con 0x7f75b015a850 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b72d4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0156380 con 0x7f75b00b4270 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 0x7f75b0154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 0x7f75b0154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51760/0 (socket says 172.21.15.83:51760) 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 -- 172.21.15.83:0/3994911249 learned_addr learned my addr 172.21.15.83:0/3994911249 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b5070700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b00b4270 0x7f75b0153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b5871700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b015a850 0x7f75b01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 -- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b00b4270 msgr2=0x7f75b0153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b00b4270 0x7f75b0153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 -- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b015a850 msgr2=0x7f75b01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b015a850 0x7f75b01554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 -- 172.21.15.83:0/3994911249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8009b30 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75b486f700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 0x7f75b0154320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c7f0 tx=0x7f75a000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0011800 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.400+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0011960 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.401+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b00cfeb0 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.401+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0011cf0 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.401+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b00cf280 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.401+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b00d0040 con 0x7f75b015a480 2023-11-28T11:10:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.402+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75a0021070 con 0x7f75b015a480 2023-11-28T11:10:31.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.402+0000 7f75a5ffb700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75940682f0 0x7f759406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.402+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f75a0085b50 con 0x7f75b015a480 2023-11-28T11:10:31.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.402+0000 7f75b5070700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75940682f0 0x7f759406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.403+0000 7f75b5070700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75940682f0 0x7f759406a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f75b00d1330 tx=0x7f75ac007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.406+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75a0052270 con 0x7f75b015a480 2023-11-28T11:10:31.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.630+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75b0074320 con 0x7f75b015a480 2023-11-28T11:10:31.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.630+0000 7f75a5ffb700 1 -- 172.21.15.83:0/3994911249 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f75a0055890 con 0x7f75b015a480 2023-11-28T11:10:31.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:31.632 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75940682f0 msgr2=0x7f759406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75940682f0 0x7f759406a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f75b00d1330 tx=0x7f75ac007ec0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 msgr2=0x7f75b0154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 0x7f75b0154320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c7f0 tx=0x7f75a000cb00 comp rx=0 tx=0).stop 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 shutdown_connections 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b015a850 0x7f75b01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75b00b4270 0x7f75b0153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75940682f0 0x7f759406a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 --2- 172.21.15.83:0/3994911249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b015a480 0x7f75b0154320 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 >> 172.21.15.83:0/3994911249 conn(0x7f75b006de20 msgr2=0x7f75b00d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:31.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 shutdown_connections 2023-11-28T11:10:31.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:31.632+0000 7f75b72d4700 1 -- 172.21.15.83:0/3994911249 wait complete. 2023-11-28T11:10:31.635 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:10:31.957 INFO:tasks.ceph.mds.d.smithi090.stderr:2023-11-28T11:10:31.957+0000 7fc27e305700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:10:32.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:10:26.382170+0000', 'modified': '2023-11-28T11:10:29.393540+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5763}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5763': {'gid': 5763, 'name': 'd', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.90:6835/3041933349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6833', 'nonce': 3041933349}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 3041933349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:10:32.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:10:32.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:10:32.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.171+0000 7f06d1286700 1 Processor -- start 2023-11-28T11:10:32.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06d1286700 1 -- start start 2023-11-28T11:10:32.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06d1286700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06cc0b4980 0x7f06cc0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06d1286700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0d0e40 0x7f06cc0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06d1286700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc0d27c0 con 0x7f06cc160bb0 2023-11-28T11:10:32.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06d1286700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc0cd8c0 con 0x7f06cc0b4980 2023-11-28T11:10:32.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06d1286700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0d2a60 con 0x7f06cc0d0e40 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.172+0000 7f06caffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06cc0b4980 0x7f06cc0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06cc0b4980 0x7f06cc0b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51766/0 (socket says 172.21.15.83:51766) 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 -- 172.21.15.83:0/1388687551 learned_addr learned my addr 172.21.15.83:0/1388687551 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1388687551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0d0e40 0x7f06cc0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 -- 172.21.15.83:0/1388687551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0d0e40 msgr2=0x7f06cc0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 --2- 172.21.15.83:0/1388687551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0d0e40 0x7f06cc0d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 -- 172.21.15.83:0/1388687551 >> v1:172.21.15.90:6789/0 conn(0x7f06cc160bb0 legacy=0x7f06cc0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 -- 172.21.15.83:0/1388687551 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc0cdba0 con 0x7f06cc0b4980 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06caffd700 1 --2- 172.21.15.83:0/1388687551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc0b4980 0x7f06cc0b5d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f06c00098f0 tx=0x7f06c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51f6228ca4fcfc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06c9ffb700 1 -- 172.21.15.83:0/1388687551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0004230 con 0x7f06cc0b4980 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06c9ffb700 1 -- 172.21.15.83:0/1388687551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c0004390 con 0x7f06cc0b4980 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.173+0000 7f06c9ffb700 1 -- 172.21.15.83:0/1388687551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c00036a0 con 0x7f06cc0b4980 2023-11-28T11:10:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 -- 172.21.15.83:0/1388687551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc0b4980 msgr2=0x7f06cc0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 --2- 172.21.15.83:0/1388687551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc0b4980 0x7f06cc0b5d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f06c00098f0 tx=0x7f06c0009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 -- 172.21.15.83:0/1388687551 shutdown_connections 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 --2- 172.21.15.83:0/1388687551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0d0e40 0x7f06cc0d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 --2- 172.21.15.83:0/1388687551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc0b4980 0x7f06cc0b5d50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 -- 172.21.15.83:0/1388687551 >> 172.21.15.83:0/1388687551 conn(0x7f06cc06ceb0 msgr2=0x7f06cc0cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 -- 172.21.15.83:0/1388687551 shutdown_connections 2023-11-28T11:10:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.174+0000 7f06d1286700 1 -- 172.21.15.83:0/1388687551 wait complete. 2023-11-28T11:10:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.175+0000 7f06d1286700 1 Processor -- start 2023-11-28T11:10:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.175+0000 7f06d1286700 1 -- start start 2023-11-28T11:10:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06d1286700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0b4980 0x7f06cc14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06d1286700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc0d0e40 0x7f06cc0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06d1286700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 0x7f06cc0b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06d1286700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc15cd70 con 0x7f06cc160bb0 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06d1286700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0cb050 con 0x7f06cc0d0e40 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06d1286700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc14b730 con 0x7f06cc0b4980 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06cb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 0x7f06cc0b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06cb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 0x7f06cc0b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51776/0 (socket says 172.21.15.83:51776) 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.176+0000 7f06cb7fe700 1 -- 172.21.15.83:0/1912067886 learned_addr learned my addr 172.21.15.83:0/1912067886 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06caffd700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0b4980 0x7f06cc14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc0d0e40 0x7f06cc0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06cb7fe700 1 -- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0b4980 msgr2=0x7f06cc14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0b4980 0x7f06cc14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06cb7fe700 1 -- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc0d0e40 msgr2=0x7f06cc0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc0d0e40 0x7f06cc0c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06cb7fe700 1 -- 172.21.15.83:0/1912067886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009550 con 0x7f06cc160bb0 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc0d0e40 0x7f06cc0c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06caffd700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0b4980 0x7f06cc14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 0x7f06cc0b3cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f06c400cc30 tx=0x7f06c400a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06b37fe700 1 -- 172.21.15.83:0/1912067886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400ab70 con 0x7f06cc160bb0 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06b37fe700 1 -- 172.21.15.83:0/1912067886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c400acd0 con 0x7f06cc160bb0 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc0cd550 con 0x7f06cc160bb0 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06b37fe700 1 -- 172.21.15.83:0/1912067886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4003c40 con 0x7f06cc160bb0 2023-11-28T11:10:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.177+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc161020 con 0x7f06cc160bb0 2023-11-28T11:10:32.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.179+0000 7f06b37fe700 1 -- 172.21.15.83:0/1912067886 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f06c401f070 con 0x7f06cc160bb0 2023-11-28T11:10:32.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.179+0000 7f06b37fe700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06b80682f0 0x7f06b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.179+0000 7f06b37fe700 1 -- 172.21.15.83:0/1912067886 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f06c4085e00 con 0x7f06cc160bb0 2023-11-28T11:10:32.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.179+0000 7f06caffd700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06b80682f0 0x7f06b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.180+0000 7f06caffd700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06b80682f0 0x7f06b806a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009c40 tx=0x7f06c0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.181+0000 7f06dd40fb80 1 Processor -- start 2023-11-28T11:10:32.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.181+0000 7f06dd40fb80 1 -- start start 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee031810 0x5564ee007a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 0x5564ee031be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee04d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564edef1bc0 con 0x5564ee02f130 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564edf60350 con 0x5564ee032120 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564edf59880 con 0x5564ee031810 2023-11-28T11:10:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 0x5564ee031be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 0x5564ee031be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51782/0 (socket says 172.21.15.83:51782) 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 -- 172.21.15.83:0/2400996746 learned_addr learned my addr 172.21.15.83:0/2400996746 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06caffd700 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee031810 0x5564ee007a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee04d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 -- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee031810 msgr2=0x5564ee007a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee031810 0x5564ee007a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 -- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 msgr2=0x5564ee04d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee04d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 -- 172.21.15.83:0/2400996746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564edddb5e0 con 0x5564ee02f130 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 0x5564ee031be0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00a8d0 tx=0x7f06bc00abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cfac0e69159c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.182+0000 7f06b0ff9700 1 -- 172.21.15.83:0/2400996746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00c890 con 0x5564ee02f130 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06b0ff9700 1 -- 172.21.15.83:0/2400996746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc00c9f0 con 0x5564ee02f130 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06b0ff9700 1 -- 172.21.15.83:0/2400996746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00cda0 con 0x5564ee02f130 2023-11-28T11:10:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 -- 172.21.15.83:0/2400996746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 msgr2=0x5564ee031be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 0x5564ee031be0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00a8d0 tx=0x7f06bc00abe0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 -- 172.21.15.83:0/2400996746 shutdown_connections 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee04d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee031810 0x5564ee007a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/2400996746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee02f130 0x5564ee031be0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 -- 172.21.15.83:0/2400996746 >> 172.21.15.83:0/2400996746 conn(0x5564ee004d60 msgr2=0x5564ee0056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 -- 172.21.15.83:0/2400996746 shutdown_connections 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.183+0000 7f06dd40fb80 1 -- 172.21.15.83:0/2400996746 wait complete. 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.184+0000 7f06dd40fb80 1 -- start start 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.184+0000 7f06dd40fb80 1 Processor -- start 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee0548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee025e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee046480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564edddd8d0 con 0x5564ee031810 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ee054490 con 0x5564ee032120 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564ee044310 con 0x5564ee02f130 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 10 client.0 _ll_register_callbacks cb 0x5564eddff240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee025e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee025e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51788/0 (socket says 172.21.15.83:51788) 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1157274252 learned_addr learned my addr 172.21.15.83:0/1157274252 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06caffd700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee0548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee046480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 msgr2=0x5564ee0548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee0548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 msgr2=0x5564ee046480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee046480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1157274252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc00a530 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06caffd700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee0548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee046480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee025e70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00ac20 tx=0x7f06bc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b15cbdcbba0cb62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06a9ffb700 1 -- 172.21.15.83:0/1157274252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc01a860 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564edf27140 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06a9ffb700 1 -- 172.21.15.83:0/1157274252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f06bc01f070 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.185+0000 7f06a9ffb700 1 -- 172.21.15.83:0/1157274252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06bc01a9c0 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 10 client.5913 fetch_fsmap learned FSMap version 25 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5564edd87d90 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06a9ffb700 1 -- 172.21.15.83:0/1157274252 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564edf27140 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06a9ffb700 1 -- 172.21.15.83:0/1157274252 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f06bc01ab20 con 0x5564ee031810 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 10 client.5913 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 10 client.5913 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:10:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 20 client.5913 populate_metadata read hostname 'smithi083' 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee030940 0x5564ee050cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 4 client.5913 mds_command: new command op to 5763 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564eddddf10 con 0x5564ee030940 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06caffd700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee030940 0x5564ee050cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06a9ffb700 10 client.5913 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.186+0000 7f06caffd700 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee030940 0x5564ee050cf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9af8f99c36b5f2 server_cookie=2534e0d5d056e7ad in_seq=0 out_seq=0 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06a9ffb700 1 -- 172.21.15.83:0/1157274252 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5564eddddf10 con 0x5564ee030940 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06a9ffb700 10 client.5913 handle_command_reply: tid=0 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 client.5913 shutdown 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 msgr2=0x5564ee025e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee025e70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00ac20 tx=0x7f06bc00f000 comp rx=0 tx=0).stop 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 shutdown_connections 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee030940 0x5564ee050cf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee032120 0x5564ee046480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06a9ffb700 0 client.5913 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee0548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1157274252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee025e70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 >> 172.21.15.83:0/1157274252 conn(0x5564ee004d60 msgr2=0x5564ee007990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 shutdown_connections 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1157274252 wait complete. 2023-11-28T11:10:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.187+0000 7f06dd40fb80 20 client.5913 trim_cache size 0 max 16384 2023-11-28T11:10:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.205+0000 7f06dd40fb80 1 Processor -- start 2023-11-28T11:10:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.205+0000 7f06dd40fb80 1 -- start start 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.205+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee02f130 0x5564ee027450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee0349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee032120 0x5564ee034f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ee027990 con 0x5564ee031810 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ee031be0 con 0x5564ee02f130 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564ee02ee90 con 0x5564ee032120 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee0349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee0349e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51796/0 (socket says 172.21.15.83:51796) 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1013615653 learned_addr learned my addr 172.21.15.83:0/1013615653 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee032120 msgr2=0x5564ee034f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06caffd700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee02f130 0x5564ee027450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee032120 0x5564ee034f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee032120 0x5564ee034f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee02f130 msgr2=0x5564ee027450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee02f130 0x5564ee027450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 -- 172.21.15.83:0/1013615653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564edf20140 con 0x5564ee031810 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee032120 0x5564ee034f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06caffd700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee02f130 0x5564ee027450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee0349e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00c150 tx=0x7f06bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4786cf1caca84f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06b17fa700 1 -- 172.21.15.83:0/1013615653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc02c350 con 0x5564ee031810 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06b17fa700 1 -- 172.21.15.83:0/1013615653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc02c4b0 con 0x5564ee031810 2023-11-28T11:10:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.206+0000 7f06b17fa700 1 -- 172.21.15.83:0/1013615653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc004030 con 0x5564ee031810 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1013615653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 msgr2=0x5564ee0349e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee0349e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00c150 tx=0x7f06bc013000 comp rx=0 tx=0).stop 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1013615653 shutdown_connections 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee032120 0x5564ee034f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee02f130 0x5564ee027450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/1013615653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee031810 0x5564ee0349e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1013615653 >> 172.21.15.83:0/1013615653 conn(0x5564ee004d60 msgr2=0x5564ee052950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1013615653 shutdown_connections 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 -- 172.21.15.83:0/1013615653 wait complete. 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.207+0000 7f06dd40fb80 1 -- start start 2023-11-28T11:10:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.208+0000 7f06dd40fb80 1 Processor -- start 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee026490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee031810 0x5564ee0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 0x5564ee0af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ee006800 con 0x5564ee032120 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ee002040 con 0x5564ee031810 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564ee0269d0 con 0x5564ee02f130 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 10 client.0 _ll_register_callbacks cb 0x5564eddff240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 0x5564ee0af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 0x5564ee0af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51804/0 (socket says 172.21.15.83:51804) 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 -- 172.21.15.83:0/3244720615 learned_addr learned my addr 172.21.15.83:0/3244720615 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 -- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 msgr2=0x5564ee026490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee026490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06ca7fc700 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee031810 0x5564ee0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 -- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee031810 msgr2=0x5564ee0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee031810 0x5564ee0aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 -- 172.21.15.83:0/3244720615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564edd87d90 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06cb7fe700 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 0x5564ee0af0f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f06c4055e90 tx=0x7f06c40529a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975ea964b20e0d02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06a9ffb700 1 -- 172.21.15.83:0/3244720615 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c406d3f0 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06a9ffb700 1 -- 172.21.15.83:0/3244720615 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f06c406d030 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564edf17160 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.209+0000 7f06a9ffb700 1 -- 172.21.15.83:0/3244720615 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06c4051fd0 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 10 client.5919 fetch_fsmap learned FSMap version 25 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5564eddae200 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06a9ffb700 1 -- 172.21.15.83:0/3244720615 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564edf17160 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06a9ffb700 1 -- 172.21.15.83:0/3244720615 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f06c4052130 con 0x5564ee032120 2023-11-28T11:10:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 10 client.5919 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 10 client.5919 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 20 client.5919 populate_metadata read hostname 'smithi083' 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee045670 0x5564ee045a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 4 client.5919 mds_command: new command op to 5763 tid=0[{"prefix": "status"}] 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5564edf01170 con 0x5564ee045670 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06caffd700 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee045670 0x5564ee045a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06a9ffb700 10 client.5919 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.210+0000 7f06caffd700 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee045670 0x5564ee045a40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5651fc6867ddced9 server_cookie=96554046948b0da0 in_seq=0 out_seq=0 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06a9ffb700 1 -- 172.21.15.83:0/3244720615 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5564edf01170 con 0x5564ee045670 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06a9ffb700 10 client.5919 handle_command_reply: tid=0 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 client.5919 shutdown 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 msgr2=0x5564ee0af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 0x5564ee0af0f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f06c4055e90 tx=0x7f06c40529a0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 shutdown_connections 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564ee032120 0x5564ee0af0f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5564ee02f130 0x5564ee026490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5564ee045670 0x5564ee045a40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 --2- 172.21.15.83:0/3244720615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ee031810 0x5564ee0aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06a9ffb700 0 client.5919 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 >> 172.21.15.83:0/3244720615 conn(0x5564ee004d60 msgr2=0x5564ee052670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 shutdown_connections 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 1 -- 172.21.15.83:0/3244720615 wait complete. 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06dd40fb80 20 client.5919 trim_cache size 0 max 16384 2023-11-28T11:10:32.218 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "69424f07-94c3-4dbc-8507-3551775e0a09", 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5763, 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8274675250000003, 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 25, 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 51, 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 51, 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.188969725 2023-11-28T11:10:32.219 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:10:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06b80682f0 msgr2=0x7f06b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06b80682f0 0x7f06b806a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009c40 tx=0x7f06c0008030 comp rx=0 tx=0).stop 2023-11-28T11:10:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 msgr2=0x7f06cc0b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 0x7f06cc0b3cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f06c400cc30 tx=0x7f06c400a1f0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 shutdown_connections 2023-11-28T11:10:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f06b80682f0 0x7f06b806a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06cc160bb0 0x7f06cc0b3cf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f06cc0b4980 0x7f06cc14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 --2- 172.21.15.83:0/1912067886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc0d0e40 0x7f06cc0c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 >> 172.21.15.83:0/1912067886 conn(0x7f06cc06ceb0 msgr2=0x7f06cc0b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 shutdown_connections 2023-11-28T11:10:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.211+0000 7f06d1286700 1 -- 172.21.15.83:0/1912067886 wait complete. 2023-11-28T11:10:32.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:10:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.343+0000 7fecfb56a700 1 Processor -- start 2023-11-28T11:10:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.343+0000 7fecfb56a700 1 -- start start 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecfb56a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fecf415c910 0x7fecf40c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecfb56a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecf415cce0 0x7fecf4160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecfb56a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf4161520 con 0x7fecf406a270 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecfb56a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf40ca830 con 0x7fecf415cce0 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecfb56a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf40ca270 con 0x7fecf415c910 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecf415cce0 0x7fecf4160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecf415cce0 0x7fecf4160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51808/0 (socket says 172.21.15.83:51808) 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 -- 172.21.15.83:0/3990926530 learned_addr learned my addr 172.21.15.83:0/3990926530 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 -- 172.21.15.83:0/3990926530 >> v2:172.21.15.90:3300/0 conn(0x7fecf415c910 msgr2=0x7fecf40c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 --2- 172.21.15.83:0/3990926530 >> v2:172.21.15.90:3300/0 conn(0x7fecf415c910 0x7fecf40c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 -- 172.21.15.83:0/3990926530 >> v1:172.21.15.90:6789/0 conn(0x7fecf406a270 legacy=0x7fecf406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 -- 172.21.15.83:0/3990926530 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40ca550 con 0x7fecf415cce0 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.344+0000 7fecf8b05700 1 --2- 172.21.15.83:0/3990926530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf4160fe0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fecf4072900 tx=0x7fece4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4803d214459c463f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecebfff700 1 -- 172.21.15.83:0/3990926530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4004230 con 0x7fecf415cce0 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecebfff700 1 -- 172.21.15.83:0/3990926530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4004390 con 0x7fecf415cce0 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecebfff700 1 -- 172.21.15.83:0/3990926530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece40036a0 con 0x7fecf415cce0 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 -- 172.21.15.83:0/3990926530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 msgr2=0x7fecf4160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 --2- 172.21.15.83:0/3990926530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf4160fe0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fecf4072900 tx=0x7fece4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 -- 172.21.15.83:0/3990926530 shutdown_connections 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 --2- 172.21.15.83:0/3990926530 >> v2:172.21.15.90:3300/0 conn(0x7fecf415c910 0x7fecf40c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 --2- 172.21.15.83:0/3990926530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf4160fe0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 -- 172.21.15.83:0/3990926530 >> 172.21.15.83:0/3990926530 conn(0x7fecf40cf430 msgr2=0x7fecf40cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 -- 172.21.15.83:0/3990926530 shutdown_connections 2023-11-28T11:10:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.345+0000 7fecfb56a700 1 -- 172.21.15.83:0/3990926530 wait complete. 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.346+0000 7fecfb56a700 1 Processor -- start 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.346+0000 7fecfb56a700 1 -- start start 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecfb56a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fecf406a270 0x7fecf40ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecfb56a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecf415c910 0x7fecf41592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecfb56a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf40ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecfb56a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40ae760 con 0x7fecf415cce0 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecfb56a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40ad180 con 0x7fecf415c910 2023-11-28T11:10:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecfb56a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40ad460 con 0x7fecf406a270 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecf9b07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf40ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecf9b07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf40ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51818/0 (socket says 172.21.15.83:51818) 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecf9b07700 1 -- 172.21.15.83:0/575780497 learned_addr learned my addr 172.21.15.83:0/575780497 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecf9306700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fecf406a270 0x7fecf40ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.347+0000 7fecf8b05700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecf415c910 0x7fecf41592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecf9b07700 1 -- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fecf406a270 msgr2=0x7fecf40ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecf9b07700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fecf406a270 0x7fecf40ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecf9b07700 1 -- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecf415c910 msgr2=0x7fecf41592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecf9b07700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecf415c910 0x7fecf41592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecf9b07700 1 -- 172.21.15.83:0/575780497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4009550 con 0x7fecf415cce0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecf9b07700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf40ac270 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fecec00c7a0 tx=0x7fecec00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec003e10 con 0x7fecf415cce0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec005070 con 0x7fecf415cce0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf40cd510 con 0x7fecf415cce0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec005420 con 0x7fecf415cce0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.348+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf40cd230 con 0x7fecf415cce0 2023-11-28T11:10:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.349+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf406dc60 con 0x7fecf415cce0 2023-11-28T11:10:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.349+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fecec019030 con 0x7fecf415cce0 2023-11-28T11:10:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.350+0000 7fece9ffb700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fecd80682f0 0x7fecd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.350+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fecec0852c0 con 0x7fecf415cce0 2023-11-28T11:10:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.350+0000 7fecf9306700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fecd80682f0 0x7fecd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.350+0000 7fecf9306700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fecd80682f0 0x7fecd806a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009a10 tx=0x7fecf0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.354+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fecec051950 con 0x7fecf415cce0 2023-11-28T11:10:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.581+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fecf40cf230 con 0x7fecf415cce0 2023-11-28T11:10:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.581+0000 7fece9ffb700 1 -- 172.21.15.83:0/575780497 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fecec017070 con 0x7fecf415cce0 2023-11-28T11:10:32.583 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:10:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fecd80682f0 msgr2=0x7fecd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fecd80682f0 0x7fecd806a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009a10 tx=0x7fecf0009190 comp rx=0 tx=0).stop 2023-11-28T11:10:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 msgr2=0x7fecf40ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf40ac270 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fecec00c7a0 tx=0x7fecec00cab0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 shutdown_connections 2023-11-28T11:10:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fecd80682f0 0x7fecd806a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf415cce0 0x7fecf40ac270 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fecf406a270 0x7fecf40ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 --2- 172.21.15.83:0/575780497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecf415c910 0x7fecf41592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 >> 172.21.15.83:0/575780497 conn(0x7fecf40cf430 msgr2=0x7fecf406a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 shutdown_connections 2023-11-28T11:10:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:32.583+0000 7fecfb56a700 1 -- 172.21.15.83:0/575780497 wait complete. 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:10:32.598 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:32.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:32.599 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42241sec preferred_lft 42241sec 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::22f1:9e4c:955b:b4a/64 scope link noprefixroute 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:10:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::80fe:9fff:fe99:6cb2/64 scope link 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:10:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::1c9b:f5ff:fe74:5abe/64 scope link 2023-11-28T11:10:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.619 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:10:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:d1:52:12:18:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:10:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5cd1:52ff:fe12:1849/64 scope link 2023-11-28T11:10:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:32.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:32.620 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:10:32.681 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:10:32.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:32.683 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-28T11:10:32.741 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-28T11:10:32.742 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:10:32.742 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:10:32.742 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:10:32.742 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:10:32.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:32.742 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:10:32.742 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:10:32.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:10:32.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:10:32.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:10:32.743 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:10:32.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:32.847 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:10:32.863 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:10:32.863 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:10:32.863 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:32.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:32.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-28T11:10:32.929 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:10:32.929 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:10:32.987 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:33.043 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-28T11:10:33.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:33.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:10:33.129 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:10:33.129 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:10:33.159 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:10:33.160 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:10:33.234 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:10:33.234 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:10:33.234 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:10:33.235 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:10:33.237 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:10:33.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:33.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:33.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:33.265 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:10:33.351 INFO:teuthology.orchestra.run.smithi083.stdout:client5225 2023-11-28T11:10:33.352 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:10:33.352 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:33.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:33.369 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T11:10:33.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:33.432 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T11:10:33.491 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:33.491 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:10:33.491 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:10:33.491 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:10:33.491 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T11:10:33.491 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:10:33.597 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2023-11-28T11:10:33.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.944+0000 7f54f5d0e700 1 Processor -- start 2023-11-28T11:10:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54f5d0e700 1 -- start start 2023-11-28T11:10:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54f5d0e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f00b1720 0x7f54f00cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54f5d0e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f54f015cff0 0x7f54f00cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54f5d0e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f00cb1d0 con 0x7f54f015a3f0 2023-11-28T11:10:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54f5d0e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f00cb540 con 0x7f54f015cff0 2023-11-28T11:10:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54f5d0e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00cddb0 con 0x7f54f00b1720 2023-11-28T11:10:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54effff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f54f015cff0 0x7f54f00cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54eeffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f54f015a3f0 0x7f54f00b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49686/0 (socket says 172.21.15.83:49686) 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.945+0000 7f54eeffd700 1 -- 172.21.15.83:0/2254569167 learned_addr learned my addr 172.21.15.83:0/2254569167 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54ef7fe700 1 --2- 172.21.15.83:0/2254569167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f00b1720 0x7f54f00cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54effff700 1 -- 172.21.15.83:0/2254569167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f00b1720 msgr2=0x7f54f00cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54effff700 1 --2- 172.21.15.83:0/2254569167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f00b1720 0x7f54f00cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54effff700 1 -- 172.21.15.83:0/2254569167 >> v1:172.21.15.83:6789/0 conn(0x7f54f015a3f0 legacy=0x7f54f00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54ee7fc700 1 -- 172.21.15.83:0/2254569167 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438520684 0 0) 0x7f54f00cb1d0 con 0x7f54f015a3f0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54effff700 1 -- 172.21.15.83:0/2254569167 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f015d840 con 0x7f54f015cff0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54effff700 1 --2- 172.21.15.83:0/2254569167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f00cb960 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009d90 tx=0x7f54e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237659454fe4c49e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54ee7fc700 1 -- 172.21.15.83:0/2254569167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800e070 con 0x7f54f015cff0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54ee7fc700 1 -- 172.21.15.83:0/2254569167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e800bb20 con 0x7f54f015cff0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.946+0000 7f54ee7fc700 1 -- 172.21.15.83:0/2254569167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8012430 con 0x7f54f015cff0 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2254569167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 msgr2=0x7f54f00cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2254569167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f00cb960 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009d90 tx=0x7f54e800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2254569167 shutdown_connections 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2254569167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f00cb960 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2254569167 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f00b1720 0x7f54f00cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2254569167 >> 172.21.15.83:0/2254569167 conn(0x7f54f00cc720 msgr2=0x7f54f00ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2254569167 shutdown_connections 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.947+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2254569167 wait complete. 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.948+0000 7f54f5d0e700 1 Processor -- start 2023-11-28T11:10:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.948+0000 7f54f5d0e700 1 -- start start 2023-11-28T11:10:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54f5d0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54f00b1720 0x7f54f00b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54f5d0e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f015a3f0 0x7f54f00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54f5d0e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f0151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54f5d0e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b1500 con 0x7f54f015cff0 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54f5d0e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b5a30 con 0x7f54f00b1720 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54f5d0e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00c94f0 con 0x7f54f015a3f0 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f0151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f0151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51832/0 (socket says 172.21.15.83:51832) 2023-11-28T11:10:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 -- 172.21.15.83:0/2857897087 learned_addr learned my addr 172.21.15.83:0/2857897087 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54ef7fe700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54f00b1720 0x7f54f00b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54eeffd700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f015a3f0 0x7f54f00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 -- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f015a3f0 msgr2=0x7f54f00b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f015a3f0 0x7f54f00b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 -- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54f00b1720 msgr2=0x7f54f00b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54f00b1720 0x7f54f00b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 -- 172.21.15.83:0/2857897087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8009a70 con 0x7f54f015cff0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.949+0000 7f54effff700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f0151290 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f54f006db00 tx=0x7f54e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.950+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8012430 con 0x7f54f015cff0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.950+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8020070 con 0x7f54f015cff0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.950+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8004280 con 0x7f54f015cff0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.950+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f015de00 con 0x7f54f015cff0 2023-11-28T11:10:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.950+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f015db20 con 0x7f54f015cff0 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.950+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f015a860 con 0x7f54f015cff0 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.951+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f54e802e070 con 0x7f54f015cff0 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.951+0000 7f54e7fff700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f54d00682f0 0x7f54d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.951+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f54e8022070 con 0x7f54f015cff0 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.951+0000 7f54ef7fe700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f54d00682f0 0x7f54d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.952+0000 7f54ef7fe700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f54d00682f0 0x7f54d006a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009b40 tx=0x7f54e0009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:33.956+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f54e8054080 con 0x7f54f015cff0 2023-11-28T11:10:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.183+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54f0074320 con 0x7f54f015cff0 2023-11-28T11:10:34.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.184+0000 7f54e7fff700 1 -- 172.21.15.83:0/2857897087 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f54e80576a0 con 0x7f54f015cff0 2023-11-28T11:10:34.185 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:34.185 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f54d00682f0 msgr2=0x7f54d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f54d00682f0 0x7f54d006a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009b40 tx=0x7f54e0009210 comp rx=0 tx=0).stop 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 msgr2=0x7f54f0151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f0151290 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f54f006db00 tx=0x7f54e8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 shutdown_connections 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f54d00682f0 0x7f54d006a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f015cff0 0x7f54f0151290 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54f00b1720 0x7f54f00b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 --2- 172.21.15.83:0/2857897087 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54f015a3f0 0x7f54f00b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 >> 172.21.15.83:0/2857897087 conn(0x7f54f00cc720 msgr2=0x7f54f0157610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 shutdown_connections 2023-11-28T11:10:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.186+0000 7f54f5d0e700 1 -- 172.21.15.83:0/2857897087 wait complete. 2023-11-28T11:10:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:10:34.200 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:10:34.358 INFO:teuthology.orchestra.run.smithi090.stdout:{"message":"","return_code":0} 2023-11-28T11:10:34.359 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:10:34.359 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:10:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.403+0000 7f994c069080 1 Processor -- start 2023-11-28T11:10:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.403+0000 7f994c069080 1 -- start start 2023-11-28T11:10:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994c069080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e7b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994c069080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e85aa00 0x55ec4e85fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994c069080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 0x55ec4e85bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994c069080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4e6c2fe0 con 0x55ec4e860a80 2023-11-28T11:10:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994c069080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4e788350 con 0x55ec4e7896f0 2023-11-28T11:10:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994c069080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec4e7b7e30 con 0x55ec4e85aa00 2023-11-28T11:10:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994296d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 0x55ec4e85bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994216c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e7b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994296d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 0x55ec4e85bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51848/0 (socket says 172.21.15.83:51848) 2023-11-28T11:10:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994196b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e85aa00 0x55ec4e85fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994296d700 1 -- 172.21.15.83:0/2226260602 learned_addr learned my addr 172.21.15.83:0/2226260602 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994216c700 1 -- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e85aa00 msgr2=0x55ec4e85fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994216c700 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e85aa00 0x55ec4e85fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994216c700 1 -- 172.21.15.83:0/2226260602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 msgr2=0x55ec4e85bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994216c700 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 0x55ec4e85bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994216c700 1 -- 172.21.15.83:0/2226260602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938005d20 con 0x55ec4e7896f0 2023-11-28T11:10:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.404+0000 7f994296d700 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 0x55ec4e85bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994216c700 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e7b6680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9934014000 tx=0x7f993400d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=346abb7944ca16b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f9940969700 1 -- 172.21.15.83:0/2226260602 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934008a00 con 0x55ec4e7896f0 2023-11-28T11:10:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f9940969700 1 -- 172.21.15.83:0/2226260602 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9934008ba0 con 0x55ec4e7896f0 2023-11-28T11:10:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f9940969700 1 -- 172.21.15.83:0/2226260602 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993401c410 con 0x55ec4e7896f0 2023-11-28T11:10:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 -- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 msgr2=0x55ec4e7b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e7b6680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9934014000 tx=0x7f993400d000 comp rx=0 tx=0).stop 2023-11-28T11:10:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 -- 172.21.15.83:0/2226260602 shutdown_connections 2023-11-28T11:10:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e860a80 0x55ec4e85bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e7b6680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 --2- 172.21.15.83:0/2226260602 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e85aa00 0x55ec4e85fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 -- 172.21.15.83:0/2226260602 >> 172.21.15.83:0/2226260602 conn(0x55ec4e7d4370 msgr2=0x55ec4e85df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 -- 172.21.15.83:0/2226260602 shutdown_connections 2023-11-28T11:10:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.405+0000 7f994c069080 1 -- 172.21.15.83:0/2226260602 wait complete. 2023-11-28T11:10:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.406+0000 7f994c069080 1 Processor -- start 2023-11-28T11:10:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.406+0000 7f994c069080 1 -- start start 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994c069080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e859950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994c069080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e85aa00 0x55ec4e787360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994c069080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e860a80 0x55ec4e786060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994c069080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4e859530 con 0x55ec4e85aa00 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994c069080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4e858f20 con 0x55ec4e7896f0 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994c069080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec4e857430 con 0x55ec4e860a80 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994216c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e859950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994216c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e859950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:38844/0 (socket says 172.21.15.83:38844) 2023-11-28T11:10:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994216c700 1 -- 172.21.15.83:0/2475318707 learned_addr learned my addr 172.21.15.83:0/2475318707 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.407+0000 7f994196b700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e85aa00 0x55ec4e787360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994296d700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e860a80 0x55ec4e786060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994216c700 1 -- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e860a80 msgr2=0x55ec4e786060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994216c700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55ec4e860a80 0x55ec4e786060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994216c700 1 -- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e85aa00 msgr2=0x55ec4e787360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994216c700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e85aa00 0x55ec4e787360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994216c700 1 -- 172.21.15.83:0/2475318707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c001e10 con 0x55ec4e7896f0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994196b700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e85aa00 0x55ec4e787360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994216c700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e859950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f9934002fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f99337fe700 1 -- 172.21.15.83:0/2475318707 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993401c820 con 0x55ec4e7896f0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f99337fe700 1 -- 172.21.15.83:0/2475318707 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993401c9a0 con 0x55ec4e7896f0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec4e6f54f0 con 0x55ec4e7896f0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f99337fe700 1 -- 172.21.15.83:0/2475318707 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993401cd50 con 0x55ec4e7896f0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.408+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec4e6ce830 con 0x55ec4e7896f0 2023-11-28T11:10:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f99337fe700 1 -- 172.21.15.83:0/2475318707 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9934036070 con 0x55ec4e7896f0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f99337fe700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9924066c10 0x7f99240690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f99337fe700 1 -- 172.21.15.83:0/2475318707 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9934087020 con 0x55ec4e7896f0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f994196b700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9924066c10 0x7f99240690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f994c069080 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55ec4e8486f0 0x55ec4e85c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x55ec4e85fec0 con 0x55ec4e8486f0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.410+0000 7f994296d700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55ec4e8486f0 0x55ec4e85c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994196b700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9924066c10 0x7f99240690c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55ec4e7887a0 tx=0x7f993800c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994296d700 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55ec4e8486f0 0x55ec4e85c2f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994296d700 1 -- 172.21.15.83:0/2475318707 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f993c0044c0 con 0x55ec4e8486f0 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55ec4e8486f0 msgr2=0x55ec4e85c2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55ec4e8486f0 0x55ec4e85c2f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9924066c10 msgr2=0x7f99240690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9924066c10 0x7f99240690c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55ec4e7887a0 tx=0x7f993800c040 comp rx=0 tx=0).stop 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 msgr2=0x55ec4e859950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 --2- 172.21.15.83:0/2475318707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec4e7896f0 0x55ec4e859950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f9934002fb0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994296d700 1 -- 172.21.15.83:0/2475318707 reap_dead start 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 shutdown_connections 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 >> 172.21.15.83:0/2475318707 conn(0x55ec4e7d4370 msgr2=0x55ec4e85dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 shutdown_connections 2023-11-28T11:10:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.411+0000 7f994c069080 1 -- 172.21.15.83:0/2475318707 wait complete. 2023-11-28T11:10:34.425 INFO:teuthology.orchestra.run.smithi083.stdout:Yalphaparent_a 2023-11-28T11:10:34.425 DEBUG:teuthology.orchestra.run.smithi083:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout: "ancestors": [ 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1099511627776, 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "alpha", 2023-11-28T11:10:34.551 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 2 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1, 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "parent_a", 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 4 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 9, 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: "old_pools": [] 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:10:34.552 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:34.553 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T11:10:34.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.597+0000 7f4e3aa0d080 1 Processor -- start 2023-11-28T11:10:34.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.597+0000 7f4e3aa0d080 1 -- start start 2023-11-28T11:10:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e3aa0d080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e20003ab0 0x55df23e97750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e3aa0d080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 0x55df23e97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e3aa0d080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55df23eb0340 0x55df23e981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e3aa0d080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df23d8c000 con 0x55df23e95350 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e3aa0d080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df23d1efe0 con 0x7f4e20003ab0 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e3aa0d080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df23eb25f0 con 0x55df23eb0340 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 0x55df23e97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 0x55df23e97c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51874/0 (socket says 172.21.15.83:51874) 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e2bfff700 1 -- 172.21.15.83:0/2391803361 learned_addr learned my addr 172.21.15.83:0/2391803361 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e30b10700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e20003ab0 0x55df23e97750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e2bfff700 1 -- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55df23eb0340 msgr2=0x55df23e981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.598+0000 7f4e31311700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55df23eb0340 0x55df23e981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2bfff700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55df23eb0340 0x55df23e981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2bfff700 1 -- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e20003ab0 msgr2=0x55df23e97750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2bfff700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e20003ab0 0x55df23e97750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2bfff700 1 -- 172.21.15.83:0/2391803361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df23d37360 con 0x55df23e95350 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e30b10700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e20003ab0 0x55df23e97750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e31311700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55df23eb0340 0x55df23e981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2bfff700 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 0x55df23e97c90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4e24002e90 tx=0x7f4e2400e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af29f7f99c091b7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2affd700 1 -- 172.21.15.83:0/2391803361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400eb60 con 0x55df23e95350 2023-11-28T11:10:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2affd700 1 -- 172.21.15.83:0/2391803361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e24008030 con 0x55df23e95350 2023-11-28T11:10:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e2affd700 1 -- 172.21.15.83:0/2391803361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400eec0 con 0x55df23e95350 2023-11-28T11:10:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/2391803361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 msgr2=0x55df23e97c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 0x55df23e97c90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4e24002e90 tx=0x7f4e2400e5b0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/2391803361 shutdown_connections 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55df23eb0340 0x55df23e981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e20003ab0 0x55df23e97750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/2391803361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e95350 0x55df23e97c90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/2391803361 >> 172.21.15.83:0/2391803361 conn(0x55df23e2fb00 msgr2=0x55df23e0e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/2391803361 shutdown_connections 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.599+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/2391803361 wait complete. 2023-11-28T11:10:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.600+0000 7f4e3aa0d080 1 Processor -- start 2023-11-28T11:10:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.600+0000 7f4e3aa0d080 1 -- start start 2023-11-28T11:10:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55df23eb0340 0x55df23f539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e20003ab0 0x55df23eacda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e92340 0x55df23ead2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df23ea7530 con 0x55df23e92340 2023-11-28T11:10:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df23ea76e0 con 0x55df23eb0340 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df23ea79c0 con 0x7f4e20003ab0 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e92340 0x55df23ead2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e92340 0x55df23ead2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51882/0 (socket says 172.21.15.83:51882) 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 -- 172.21.15.83:0/1989280935 learned_addr learned my addr 172.21.15.83:0/1989280935 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e30b10700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55df23eb0340 0x55df23f539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e2bfff700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e20003ab0 0x55df23eacda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 -- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e20003ab0 msgr2=0x55df23eacda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e20003ab0 0x55df23eacda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 -- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55df23eb0340 msgr2=0x55df23f539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55df23eb0340 0x55df23f539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 -- 172.21.15.83:0/1989280935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2400a040 con 0x55df23e92340 2023-11-28T11:10:34.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e31311700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e92340 0x55df23ead2e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55df23e0df10 tx=0x7f4e2c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e29ffb700 1 -- 172.21.15.83:0/1989280935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c012450 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e29ffb700 1 -- 172.21.15.83:0/1989280935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c01f070 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55df23ebb020 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e29ffb700 1 -- 172.21.15.83:0/1989280935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c01ba70 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.601+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df23d2a830 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e29ffb700 1 -- 172.21.15.83:0/1989280935 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e2c01bbd0 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e29ffb700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e14066c10 0x7f4e140690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e29ffb700 1 -- 172.21.15.83:0/1989280935 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4e2c021070 con 0x55df23e92340 2023-11-28T11:10:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e30b10700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e14066c10 0x7f4e140690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55df23de69f0 0x55df23f5dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) v8 -- 0x55df23e0fb30 con 0x55df23de69f0 2023-11-28T11:10:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.603+0000 7f4e30b10700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e14066c10 0x7f4e140690c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55df23eb3300 tx=0x7f4e1c000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.604+0000 7f4e2bfff700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55df23de69f0 0x55df23f5dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.604+0000 7f4e2bfff700 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55df23de69f0 0x55df23f5dd10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e2bfff700 1 -- 172.21.15.83:0/1989280935 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4e24024040 con 0x55df23de69f0 2023-11-28T11:10:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55df23de69f0 msgr2=0x55df23f5dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55df23de69f0 0x55df23f5dd10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e14066c10 msgr2=0x7f4e140690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e14066c10 0x7f4e140690c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55df23eb3300 tx=0x7f4e1c000ed0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e92340 msgr2=0x55df23ead2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 --2- 172.21.15.83:0/1989280935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df23e92340 0x55df23ead2e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55df23e0df10 tx=0x7f4e2c005d80 comp rx=0 tx=0).stop 2023-11-28T11:10:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e31311700 1 -- 172.21.15.83:0/1989280935 reap_dead start 2023-11-28T11:10:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 shutdown_connections 2023-11-28T11:10:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 >> 172.21.15.83:0/1989280935 conn(0x55df23e2fb00 msgr2=0x55df23de4650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 shutdown_connections 2023-11-28T11:10:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.605+0000 7f4e3aa0d080 1 -- 172.21.15.83:0/1989280935 wait complete. 2023-11-28T11:10:34.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T11:10:34.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T11:10:34.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.744+0000 7f7134eab700 1 Processor -- start 2023-11-28T11:10:34.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.744+0000 7f7134eab700 1 -- start start 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f7134eab700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71300b2230 0x7f713006ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f7134eab700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300d2760 0x7f71300b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f7134eab700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71300b52a0 con 0x7f71300b4e80 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f7134eab700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300cdb20 con 0x7f71300b2230 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f7134eab700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300b2600 con 0x7f71300d2760 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71300b2230 0x7f713006ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71300b2230 0x7f713006ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51888/0 (socket says 172.21.15.83:51888) 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 -- 172.21.15.83:0/315551109 learned_addr learned my addr 172.21.15.83:0/315551109 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712dd9b700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300d2760 0x7f71300b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 -- 172.21.15.83:0/315551109 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300d2760 msgr2=0x7f71300b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300d2760 0x7f71300b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 -- 172.21.15.83:0/315551109 >> v1:172.21.15.90:6789/0 conn(0x7f71300b4e80 legacy=0x7f71301601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 -- 172.21.15.83:0/315551109 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300b5cf0 con 0x7f71300b2230 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712dd9b700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300d2760 0x7f71300b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712e59c700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b2230 0x7f713006ac80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f71200098f0 tx=0x7f7120009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b8973e10c234f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712d59a700 1 -- 172.21.15.83:0/315551109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000b8a0 con 0x7f71300b2230 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712d59a700 1 -- 172.21.15.83:0/315551109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000ba00 con 0x7f71300b2230 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.745+0000 7f712d59a700 1 -- 172.21.15.83:0/315551109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000bd90 con 0x7f71300b2230 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 -- 172.21.15.83:0/315551109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b2230 msgr2=0x7f713006ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b2230 0x7f713006ac80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f71200098f0 tx=0x7f7120009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 -- 172.21.15.83:0/315551109 shutdown_connections 2023-11-28T11:10:34.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b2230 0x7f713006ac80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 --2- 172.21.15.83:0/315551109 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300d2760 0x7f71300b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 -- 172.21.15.83:0/315551109 >> 172.21.15.83:0/315551109 conn(0x7f71300d30a0 msgr2=0x7f71300d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 -- 172.21.15.83:0/315551109 shutdown_connections 2023-11-28T11:10:34.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.746+0000 7f7134eab700 1 -- 172.21.15.83:0/315551109 wait complete. 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 Processor -- start 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 -- start start 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300b2230 0x7f71301f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 0x7f7130155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71300d2760 0x7f71300b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713015ede0 con 0x7f71300b4e80 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71301f6820 con 0x7f71300d2760 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f7134eab700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71301f30b0 con 0x7f71300b2230 2023-11-28T11:10:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f712dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 0x7f7130155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f712dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 0x7f7130155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51896/0 (socket says 172.21.15.83:51896) 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.748+0000 7f712dd9b700 1 -- 172.21.15.83:0/3975664645 learned_addr learned my addr 172.21.15.83:0/3975664645 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712ed9d700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71300d2760 0x7f71300b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712e59c700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300b2230 0x7f71301f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712dd9b700 1 -- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300b2230 msgr2=0x7f71301f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712dd9b700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300b2230 0x7f71301f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712dd9b700 1 -- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71300d2760 msgr2=0x7f71300b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712dd9b700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71300d2760 0x7f71300b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712dd9b700 1 -- 172.21.15.83:0/3975664645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120009550 con 0x7f71300b4e80 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712ed9d700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71300d2760 0x7f71300b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f712dd9b700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 0x7f7130155290 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f712400e810 tx=0x7f712400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f711effd700 1 -- 172.21.15.83:0/3975664645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400c020 con 0x7f71300b4e80 2023-11-28T11:10:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f711effd700 1 -- 172.21.15.83:0/3975664645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124004250 con 0x7f71300b4e80 2023-11-28T11:10:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f711effd700 1 -- 172.21.15.83:0/3975664645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240036a0 con 0x7f71300b4e80 2023-11-28T11:10:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300cf080 con 0x7f71300b4e80 2023-11-28T11:10:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.749+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300cd480 con 0x7f71300b4e80 2023-11-28T11:10:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.751+0000 7f711effd700 1 -- 172.21.15.83:0/3975664645 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7124003800 con 0x7f71300b4e80 2023-11-28T11:10:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.751+0000 7f711effd700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71100682f0 0x7f711006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.751+0000 7f711effd700 1 -- 172.21.15.83:0/3975664645 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7124085b50 con 0x7f71300b4e80 2023-11-28T11:10:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.751+0000 7f712e59c700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71100682f0 0x7f711006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.752+0000 7f712e59c700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71100682f0 0x7f711006a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7120009c40 tx=0x7f7120008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.753+0000 7f7141034b80 1 Processor -- start 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.753+0000 7f7141034b80 1 -- start start 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f7141034b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 0x5621c24cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24cb1f0 0x5621c24e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f7141034b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c238ac90 con 0x5621c24c8200 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c23f9420 con 0x5621c24cb1f0 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c23f2950 con 0x5621c24ca8e0 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 0x5621c24cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 0x5621c24cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51904/0 (socket says 172.21.15.83:51904) 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 -- 172.21.15.83:0/845242371 learned_addr learned my addr 172.21.15.83:0/845242371 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712ed9d700 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24cb1f0 0x5621c24e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712e59c700 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 -- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 msgr2=0x5621c24a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 -- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24cb1f0 msgr2=0x5621c24e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24cb1f0 0x5621c24e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712dd9b700 1 -- 172.21.15.83:0/845242371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c22746b0 con 0x5621c24c8200 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.754+0000 7f712ed9d700 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24cb1f0 0x5621c24e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f712dd9b700 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 0x5621c24cacb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7124055c30 tx=0x7f7124055ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2feb164f26868ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f710ffff700 1 -- 172.21.15.83:0/845242371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124052640 con 0x5621c24c8200 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f710ffff700 1 -- 172.21.15.83:0/845242371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124059e90 con 0x5621c24c8200 2023-11-28T11:10:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f710ffff700 1 -- 172.21.15.83:0/845242371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124055620 con 0x5621c24c8200 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 -- 172.21.15.83:0/845242371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 msgr2=0x5621c24cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 0x5621c24cacb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7124055c30 tx=0x7f7124055ba0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 -- 172.21.15.83:0/845242371 shutdown_connections 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24cb1f0 0x5621c24e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 --2- 172.21.15.83:0/845242371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24c8200 0x5621c24cacb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.755+0000 7f7141034b80 1 -- 172.21.15.83:0/845242371 >> 172.21.15.83:0/845242371 conn(0x5621c249de30 msgr2=0x5621c249e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.756+0000 7f7141034b80 1 -- 172.21.15.83:0/845242371 shutdown_connections 2023-11-28T11:10:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.756+0000 7f7141034b80 1 -- 172.21.15.83:0/845242371 wait complete. 2023-11-28T11:10:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.756+0000 7f7141034b80 1 -- start start 2023-11-28T11:10:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.756+0000 7f7141034b80 1 Processor -- start 2023-11-28T11:10:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c22769a0 con 0x5621c24cb1f0 2023-11-28T11:10:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c24ce510 con 0x5621c24c8200 2023-11-28T11:10:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c24c6050 con 0x5621c24ca8e0 2023-11-28T11:10:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f7141034b80 10 client.0 _ll_register_callbacks cb 0x5621c2298310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51912/0 (socket says 172.21.15.83:51912) 2023-11-28T11:10:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f712ed9d700 1 -- 172.21.15.83:0/1108736334 learned_addr learned my addr 172.21.15.83:0/1108736334 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f712e59c700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.757+0000 7f712dd9b700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f712ed9d700 1 -- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 msgr2=0x5621c24ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f712ed9d700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f712ed9d700 1 -- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 msgr2=0x5621c24ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f712ed9d700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24ecfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f712ed9d700 1 -- 172.21.15.83:0/1108736334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300cd480 con 0x5621c24cb1f0 2023-11-28T11:10:34.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f712ed9d700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24c4e20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f712800eb50 tx=0x7f712800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed5d079f37d649b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f710d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:10:34.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f710d7fa700 1 -- 172.21.15.83:0/1108736334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128013040 con 0x5621c24cb1f0 2023-11-28T11:10:34.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f710d7fa700 1 -- 172.21.15.83:0/1108736334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f71280020a0 con 0x5621c24cb1f0 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621c23c0210 con 0x5621c24cb1f0 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f710d7fa700 1 -- 172.21.15.83:0/1108736334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7128003e10 con 0x5621c24cb1f0 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 10 client.5964 fetch_fsmap learned FSMap version 25 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5621c2220e60 con 0x5621c24cb1f0 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f710d7fa700 1 -- 172.21.15.83:0/1108736334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621c23c0210 con 0x5621c24cb1f0 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f710d7fa700 1 -- 172.21.15.83:0/1108736334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f71280050d0 con 0x5621c24cb1f0 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 10 client.5964 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 20 client.5964 populate_metadata read hostname 'smithi083' 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24c7dd0 0x5621c24c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 4 client.5964 mds_command: new command op to 5763 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.758+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621c2276fe0 con 0x5621c24c7dd0 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.759+0000 7f712e59c700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24c7dd0 0x5621c24c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.759+0000 7f710d7fa700 10 client.5964 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.759+0000 7f712e59c700 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24c7dd0 0x5621c24c5850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a167a2ccb16c8b server_cookie=eb71aa48b9021619 in_seq=0 out_seq=0 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f710d7fa700 1 -- 172.21.15.83:0/1108736334 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5621c2276fe0 con 0x5621c24c7dd0 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f710d7fa700 10 client.5964 handle_command_reply: tid=0 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 client.5964 shutdown 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 msgr2=0x5621c24c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24c4e20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f712800eb50 tx=0x7f712800ef10 comp rx=0 tx=0).stop 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 shutdown_connections 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24c4e20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24c7dd0 0x5621c24c5850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f710d7fa700 0 client.5964 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 --2- 172.21.15.83:0/1108736334 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24ca8e0 0x5621c24ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 >> 172.21.15.83:0/1108736334 conn(0x5621c249de30 msgr2=0x5621c249e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 shutdown_connections 2023-11-28T11:10:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 1 -- 172.21.15.83:0/1108736334 wait complete. 2023-11-28T11:10:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.760+0000 7f7141034b80 20 client.5964 trim_cache size 0 max 16384 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.767+0000 7f8e9079f700 1 Processor -- start 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.767+0000 7f8e9079f700 1 -- start start 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e9079f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8e881582a0 0x7f8e880b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e9079f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e880cb2e0 0x7f8e8815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e9079f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e88070430 0x7f8e880708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e9079f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e88070df0 con 0x7f8e88070430 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e9079f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e880cbb80 con 0x7f8e881582a0 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e9079f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880b05f0 con 0x7f8e880cb2e0 2023-11-28T11:10:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e88070430 0x7f8e880708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e88070430 0x7f8e880708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51920/0 (socket says 172.21.15.83:51920) 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 -- 172.21.15.83:0/4217039051 learned_addr learned my addr 172.21.15.83:0/4217039051 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8e53b700 1 --2- 172.21.15.83:0/4217039051 >> v2:172.21.15.90:3300/0 conn(0x7f8e881582a0 0x7f8e880b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8dd3a700 1 --2- 172.21.15.83:0/4217039051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e880cb2e0 0x7f8e8815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 -- 172.21.15.83:0/4217039051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e880cb2e0 msgr2=0x7f8e8815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 --2- 172.21.15.83:0/4217039051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e880cb2e0 0x7f8e8815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 -- 172.21.15.83:0/4217039051 >> v2:172.21.15.90:3300/0 conn(0x7f8e881582a0 msgr2=0x7f8e880b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 --2- 172.21.15.83:0/4217039051 >> v2:172.21.15.90:3300/0 conn(0x7f8e881582a0 0x7f8e880b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 -- 172.21.15.83:0/4217039051 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8815ba50 con 0x7f8e88070430 2023-11-28T11:10:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.768+0000 7f8e8ed3c700 1 --2- 172.21.15.83:0/4217039051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e880708b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e880d1d90 tx=0x7f8e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa7b49425e4ab05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e8d539700 1 -- 172.21.15.83:0/4217039051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000e070 con 0x7f8e88070430 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e8d539700 1 -- 172.21.15.83:0/4217039051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000b9c0 con 0x7f8e88070430 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e8d539700 1 -- 172.21.15.83:0/4217039051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000bd70 con 0x7f8e88070430 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 -- 172.21.15.83:0/4217039051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 msgr2=0x7f8e880708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 --2- 172.21.15.83:0/4217039051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e880708b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e880d1d90 tx=0x7f8e8000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 -- 172.21.15.83:0/4217039051 shutdown_connections 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 --2- 172.21.15.83:0/4217039051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e880708b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 --2- 172.21.15.83:0/4217039051 >> v2:172.21.15.90:3300/0 conn(0x7f8e881582a0 0x7f8e880b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 --2- 172.21.15.83:0/4217039051 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e880cb2e0 0x7f8e8815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 -- 172.21.15.83:0/4217039051 >> 172.21.15.83:0/4217039051 conn(0x7f8e880d0c90 msgr2=0x7f8e880d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 -- 172.21.15.83:0/4217039051 shutdown_connections 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.769+0000 7f8e9079f700 1 -- 172.21.15.83:0/4217039051 wait complete. 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.770+0000 7f8e9079f700 1 Processor -- start 2023-11-28T11:10:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.770+0000 7f8e9079f700 1 -- start start 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e9079f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e881561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e9079f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e880cb2e0 0x7f8e880c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e9079f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e881582a0 0x7f8e880b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e9079f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880c9c60 con 0x7f8e88070430 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e9079f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88156710 con 0x7f8e880cb2e0 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e9079f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881e3da0 con 0x7f8e881582a0 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e8e53b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e881561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e8e53b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e881561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51922/0 (socket says 172.21.15.83:51922) 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e8e53b700 1 -- 172.21.15.83:0/4291462369 learned_addr learned my addr 172.21.15.83:0/4291462369 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e8dd3a700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e880cb2e0 0x7f8e880c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.771+0000 7f8e8ed3c700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e881582a0 0x7f8e880b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8e53b700 1 -- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e881582a0 msgr2=0x7f8e880b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8e53b700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e881582a0 0x7f8e880b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8e53b700 1 -- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e880cb2e0 msgr2=0x7f8e880c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8e53b700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e880cb2e0 0x7f8e880c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8e53b700 1 -- 172.21.15.83:0/4291462369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009b30 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8ed3c700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8e881582a0 0x7f8e880b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8dd3a700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e880cb2e0 0x7f8e880c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e8e53b700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e88070430 0x7f8e881561d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400d830 tx=0x7f8e8400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840125f0 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e84012750 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e9079f700 1 -- 172.21.15.83:0/4291462369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e880b0840 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84012b00 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e9079f700 1 -- 172.21.15.83:0/4291462369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e880cb8a0 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.772+0000 7f8e9079f700 1 -- 172.21.15.83:0/4291462369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8e880b0b20 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.773+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e84012c60 con 0x7f8e88070430 2023-11-28T11:10:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.774+0000 7f8e7effd700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e6c0682f0 0x7f8e6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.774+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8e84086c60 con 0x7f8e88070430 2023-11-28T11:10:34.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.774+0000 7f8e8dd3a700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e6c0682f0 0x7f8e6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.774+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7f8e840870e0 con 0x7f8e88070430 2023-11-28T11:10:34.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.774+0000 7f8e9079f700 1 -- 172.21.15.83:0/4291462369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f8e88157110 con 0x7f8e88070430 2023-11-28T11:10:34.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.774+0000 7f8e8dd3a700 1 --2- 172.21.15.83:0/4291462369 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8e6c0682f0 0x7f8e6c06a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e880c8e90 tx=0x7f8e78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.778+0000 7f7141034b80 1 Processor -- start 2023-11-28T11:10:34.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 -- start start 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24c8200 0x5621c2499520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24ca8e0 0x5621c249e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24f7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c2495b30 con 0x5621c24cb1f0 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c249c690 con 0x5621c24ca8e0 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c2499a60 con 0x5621c24c8200 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24f7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51934/0 (socket says 172.21.15.83:51934) 2023-11-28T11:10:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f712ed9d700 1 -- 172.21.15.83:0/183340211 learned_addr learned my addr 172.21.15.83:0/183340211 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.779+0000 7f712dd9b700 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24ca8e0 0x5621c249e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712e59c700 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24c8200 0x5621c2499520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712ed9d700 1 -- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24c8200 msgr2=0x5621c2499520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712ed9d700 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24c8200 0x5621c2499520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712ed9d700 1 -- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24ca8e0 msgr2=0x5621c249e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712ed9d700 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24ca8e0 0x5621c249e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712ed9d700 1 -- 172.21.15.83:0/183340211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c23b9210 con 0x5621c24cb1f0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712dd9b700 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24ca8e0 0x5621c249e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f712ed9d700 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24f7980 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7128001890 tx=0x7f7128011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb6e5c37a72c946 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f711cff9700 1 -- 172.21.15.83:0/183340211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128013040 con 0x5621c24cb1f0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f711cff9700 1 -- 172.21.15.83:0/183340211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128005680 con 0x5621c24cb1f0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.780+0000 7f711cff9700 1 -- 172.21.15.83:0/183340211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128005890 con 0x5621c24cb1f0 2023-11-28T11:10:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 -- 172.21.15.83:0/183340211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 msgr2=0x5621c24f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24f7980 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7128001890 tx=0x7f7128011000 comp rx=0 tx=0).stop 2023-11-28T11:10:34.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 -- 172.21.15.83:0/183340211 shutdown_connections 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24cb1f0 0x5621c24f7980 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24c8200 0x5621c2499520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 --2- 172.21.15.83:0/183340211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24ca8e0 0x5621c249e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 -- 172.21.15.83:0/183340211 >> 172.21.15.83:0/183340211 conn(0x5621c249de30 msgr2=0x5621c24e9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 -- 172.21.15.83:0/183340211 shutdown_connections 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 -- 172.21.15.83:0/183340211 wait complete. 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 -- start start 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.781+0000 7f7141034b80 1 Processor -- start 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 0x5621c2530eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24cb1f0 0x5621c24c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c24c6220 con 0x5621c24ca8e0 2023-11-28T11:10:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c25313f0 con 0x5621c24c8200 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c24f6cb0 con 0x5621c24cb1f0 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f7141034b80 10 client.0 _ll_register_callbacks cb 0x5621c2298310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f712dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 0x5621c2530eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f712dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 0x5621c2530eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51938/0 (socket says 172.21.15.83:51938) 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f712dd9b700 1 -- 172.21.15.83:0/3062169450 learned_addr learned my addr 172.21.15.83:0/3062169450 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f712e59c700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.782+0000 7f712ed9d700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24cb1f0 0x5621c24c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712dd9b700 1 -- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24cb1f0 msgr2=0x5621c24c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712dd9b700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24cb1f0 0x5621c24c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712dd9b700 1 -- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 msgr2=0x5621c24f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712dd9b700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712dd9b700 1 -- 172.21.15.83:0/3062169450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621c2220e60 con 0x5621c24ca8e0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712dd9b700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 0x5621c2530eb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f712405e180 tx=0x7f712405b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43107f6ad4236c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f710d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f710d7fa700 1 -- 172.21.15.83:0/3062169450 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712405db30 con 0x5621c24ca8e0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f710d7fa700 1 -- 172.21.15.83:0/3062169450 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f712405dc90 con 0x5621c24ca8e0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621c23b0230 con 0x5621c24ca8e0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f710d7fa700 1 -- 172.21.15.83:0/3062169450 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7124069120 con 0x5621c24ca8e0 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 10 client.5976 fetch_fsmap learned FSMap version 25 2023-11-28T11:10:34.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5621c22472d0 con 0x5621c24ca8e0 2023-11-28T11:10:34.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f710d7fa700 1 -- 172.21.15.83:0/3062169450 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621c23b0230 con 0x5621c24ca8e0 2023-11-28T11:10:34.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f710d7fa700 1 -- 172.21.15.83:0/3062169450 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7124069280 con 0x5621c24ca8e0 2023-11-28T11:10:34.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 10 client.5976 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:10:34.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 10 client.5976 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:34.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 20 client.5976 populate_metadata read hostname 'smithi083' 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24cc480 0x5621c24cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 4 client.5976 mds_command: new command op to 5763 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5621c239a240 con 0x5621c24cc480 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.783+0000 7f712e59c700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24cc480 0x5621c24cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.784+0000 7f710d7fa700 10 client.5976 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.784+0000 7f712e59c700 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24cc480 0x5621c24cc850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec4b72a0e9a2030 server_cookie=216ede4a65ccc4fb in_seq=0 out_seq=0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.784+0000 7f710d7fa700 1 -- 172.21.15.83:0/3062169450 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5621c239a240 con 0x5621c24cc480 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.784+0000 7f710d7fa700 10 client.5976 handle_command_reply: tid=0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.784+0000 7f7141034b80 1 client.5976 shutdown 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 msgr2=0x5621c2530eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 0x5621c2530eb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f712405e180 tx=0x7f712405b070 comp rx=0 tx=0).stop 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 shutdown_connections 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5621c24cb1f0 0x5621c24c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5621c24cc480 0x5621c24cc850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621c24c8200 0x5621c24f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 --2- 172.21.15.83:0/3062169450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c24ca8e0 0x5621c2530eb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f710d7fa700 0 client.5976 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 >> 172.21.15.83:0/3062169450 conn(0x5621c249de30 msgr2=0x5621c24e9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 shutdown_connections 2023-11-28T11:10:34.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 1 -- 172.21.15.83:0/3062169450 wait complete. 2023-11-28T11:10:34.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7141034b80 20 client.5976 trim_cache size 0 max 16384 2023-11-28T11:10:34.800 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:10:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2023-11-28T11:10:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "261043a7-cd1e-4c3b-b658-0c6433ba5db3", 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:10:34.804 INFO:tasks.ceph.mds.d.smithi090.stderr:2023-11-28T11:10:34.785+0000 7fc27aafe700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5225=0-4194304@1} caps={5225=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5225 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b2fc5efb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:10:34.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71100682f0 msgr2=0x7f711006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71100682f0 0x7f711006a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7120009c40 tx=0x7f7120008030 comp rx=0 tx=0).stop 2023-11-28T11:10:34.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 msgr2=0x7f7130155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 0x7f7130155290 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f712400e810 tx=0x7f712400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 shutdown_connections 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71300d2760 0x7f71300b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71300b2230 0x7f71301f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71100682f0 0x7f711006a7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 --2- 172.21.15.83:0/3975664645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300b4e80 0x7f7130155290 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 >> 172.21.15.83:0/3975664645 conn(0x7f71300d30a0 msgr2=0x7f71300b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 shutdown_connections 2023-11-28T11:10:34.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.785+0000 7f7134eab700 1 -- 172.21.15.83:0/3975664645 wait complete. 2023-11-28T11:10:34.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:10:34.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.927+0000 7fd9c77ae700 1 Processor -- start 2023-11-28T11:10:34.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.927+0000 7fd9c77ae700 1 -- start start 2023-11-28T11:10:34.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c77ae700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 0x7fd9c00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c77ae700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c006d4a0 con 0x7fd9c015c890 2023-11-28T11:10:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c77ae700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00b5ae0 con 0x7fd9c006e370 2023-11-28T11:10:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c77ae700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006c400 con 0x7fd9c00aee60 2023-11-28T11:10:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c554a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd9c015c890 0x7fd9c006dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49804/0 (socket says 172.21.15.83:49804) 2023-11-28T11:10:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c554a700 1 -- 172.21.15.83:0/1109555846 learned_addr learned my addr 172.21.15.83:0/1109555846 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:10:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 0x7fd9c00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955996120 0 0) 0x7fd9c006d4a0 con 0x7fd9c015c890 2023-11-28T11:10:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.928+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ac0035b0 con 0x7fd9c015c890 2023-11-28T11:10:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821502033 0 0) 0x7fd9c00b5ae0 con 0x7fd9c006e370 2023-11-28T11:10:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c006d4a0 con 0x7fd9c006e370 2023-11-28T11:10:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352071242 0 0) 0x7fd9ac0035b0 con 0x7fd9c015c890 2023-11-28T11:10:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 msgr2=0x7fd9c00b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 0x7fd9c00b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 >> v1:172.21.15.90:6789/0 conn(0x7fd9c006e370 legacy=0x7fd9c006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00b5240 con 0x7fd9c015c890 2023-11-28T11:10:34.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fd9b00017f0 con 0x7fd9c015c890 2023-11-28T11:10:34.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 >> v1:172.21.15.83:6789/0 conn(0x7fd9c015c890 legacy=0x7fd9c006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ac003bc0 0x7fd9ac003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 0x7fd9ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac004d90 0x7fd9ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006c400 con 0x7fd9ac004d90 2023-11-28T11:10:34.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac00bb40 con 0x7fd9ac003bc0 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac00bd00 con 0x7fd9c00aee60 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ac003bc0 0x7fd9ac003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac004d90 0x7fd9ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 0x7fd9ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 -- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 msgr2=0x7fd9ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9c00aee60 0x7fd9ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 -- 172.21.15.83:0/1109555846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac004d90 msgr2=0x7fd9ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac004d90 0x7fd9ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 -- 172.21.15.83:0/1109555846 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0035b0 con 0x7fd9ac003bc0 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac004d90 0x7fd9ac005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.929+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ac003bc0 0x7fd9ac003fb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c240 tx=0x7fd9ac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b50f1afb128830 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000cd80 con 0x7fd9ac003bc0 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0007b50 con 0x7fd9ac003bc0 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9b7fff700 1 -- 172.21.15.83:0/1109555846 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0007310 con 0x7fd9ac003bc0 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c77ae700 1 -- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ac003bc0 msgr2=0x7fd9ac003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/1109555846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ac003bc0 0x7fd9ac003fb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c240 tx=0x7fd9ac005880 comp rx=0 tx=0).stop 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/1109555846 reap_dead start 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c77ae700 1 -- 172.21.15.83:0/1109555846 shutdown_connections 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c77ae700 1 -- 172.21.15.83:0/1109555846 >> 172.21.15.83:0/1109555846 conn(0x7fd9c00b2b60 msgr2=0x7fd9c00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c77ae700 1 -- 172.21.15.83:0/1109555846 shutdown_connections 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.930+0000 7fd9c77ae700 1 -- 172.21.15.83:0/1109555846 wait complete. 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.931+0000 7fd9c77ae700 1 Processor -- start 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.931+0000 7fd9c77ae700 1 -- start start 2023-11-28T11:10:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c77ae700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9ac003bc0 0x7fd9c015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c77ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9c0149f50 0x7fd9c015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c77ae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 0x7fd9c015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c77ae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c015dc30 con 0x7fd9c015f580 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c77ae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00b0120 con 0x7fd9c0149f50 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c77ae700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00b0400 con 0x7fd9ac003bc0 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c5d4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 0x7fd9c015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c5d4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 0x7fd9c015f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51960/0 (socket says 172.21.15.83:51960) 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/378242391 learned_addr learned my addr 172.21.15.83:0/378242391 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.932+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9c0149f50 0x7fd9c015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c554a700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9ac003bc0 0x7fd9c015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9ac003bc0 msgr2=0x7fd9c015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9ac003bc0 0x7fd9c015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9c0149f50 msgr2=0x7fd9c015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9c0149f50 0x7fd9c015f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/378242391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00b5800 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9c0149f50 0x7fd9c015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 0x7fd9c015f9f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0070610 tx=0x7fd9b800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9b5ffb700 1 -- 172.21.15.83:0/378242391 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b801c070 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9b5ffb700 1 -- 172.21.15.83:0/378242391 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b800f040 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9b5ffb700 1 -- 172.21.15.83:0/378242391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800e040 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c00b5520 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.933+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c006ac30 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.935+0000 7fd9b5ffb700 1 -- 172.21.15.83:0/378242391 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd9b8015430 con 0x7fd9c015f580 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.935+0000 7fd9b5ffb700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd9a40682f0 0x7fd9a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.935+0000 7fd9b5ffb700 1 -- 172.21.15.83:0/378242391 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd9b8085fb0 con 0x7fd9c015f580 2023-11-28T11:10:34.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.935+0000 7fd9c554a700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd9a40682f0 0x7fd9a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.935+0000 7fd9c554a700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd9a40682f0 0x7fd9a406a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c240 tx=0x7fd9b000a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.937+0000 7fd9d3937b80 1 Processor -- start 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.937+0000 7fd9d3937b80 1 -- start start 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x56463901fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 0x564639049c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x5646390657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564638f09c10 con 0x564639047180 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564638f783a0 con 0x56463904a170 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x564638f718d0 con 0x564639049860 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c4d49700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 0x564639049c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c4d49700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 0x564639049c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51966/0 (socket says 172.21.15.83:51966) 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c4d49700 1 -- 172.21.15.83:0/608005621 learned_addr learned my addr 172.21.15.83:0/608005621 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x5646390657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c554a700 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x56463901fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 msgr2=0x56463901fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x56463901fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/608005621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 msgr2=0x564639049c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 0x564639049c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/608005621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564638df3630 con 0x56463904a170 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 0x564639049c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x5646390657c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80562c0 tx=0x7fd9b8055da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb0229a00a2852 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9a2ffd700 1 -- 172.21.15.83:0/608005621 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80546d0 con 0x56463904a170 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9a2ffd700 1 -- 172.21.15.83:0/608005621 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b805df90 con 0x56463904a170 2023-11-28T11:10:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.938+0000 7fd9a2ffd700 1 -- 172.21.15.83:0/608005621 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b801fbf0 con 0x56463904a170 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 -- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 msgr2=0x5646390657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x5646390657c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80562c0 tx=0x7fd9b8055da0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 -- 172.21.15.83:0/608005621 shutdown_connections 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x5646390657c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x56463901fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/608005621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639047180 0x564639049c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 -- 172.21.15.83:0/608005621 >> 172.21.15.83:0/608005621 conn(0x56463901cdb0 msgr2=0x56463901d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 -- 172.21.15.83:0/608005621 shutdown_connections 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.939+0000 7fd9d3937b80 1 -- 172.21.15.83:0/608005621 wait complete. 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.940+0000 7fd9d3937b80 1 -- start start 2023-11-28T11:10:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.940+0000 7fd9d3937b80 1 Processor -- start 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 0x564639067b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x5646390680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564638df5920 con 0x56463904a170 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564639069440 con 0x564639049860 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56463906cb10 con 0x564639047180 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9d3937b80 10 client.0 _ll_register_callbacks cb 0x564638e17290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c5d4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x5646390680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 0x564639067b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c554a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 0x564639067b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:38964/0 (socket says 172.21.15.83:38964) 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c5d4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x5646390680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51974/0 (socket says 172.21.15.83:51974) 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 -- 172.21.15.83:0/1279992687 learned_addr learned my addr 172.21.15.83:0/1279992687 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 -- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 msgr2=0x5646390404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 -- 172.21.15.83:0/1279992687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 msgr2=0x5646390680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x5646390680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c4d49700 1 -- 172.21.15.83:0/1279992687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c006ac30 con 0x564639049860 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.941+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x5646390680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 0x564639067b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc003bf0 tx=0x7fd9bc003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43f3de4d6f78dcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd99bfff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd99bfff700 1 -- 172.21.15.83:0/1279992687 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc006860 con 0x564639049860 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd99bfff700 1 -- 172.21.15.83:0/1279992687 <== mon.1 v2:172.21.15.90:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd9bc0069c0 con 0x564639049860 2023-11-28T11:10:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564638f3f190 con 0x564639049860 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd99bfff700 1 -- 172.21.15.83:0/1279992687 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9bc006b80 con 0x564639049860 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 10 client.5572 fetch_fsmap learned FSMap version 25 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564638d9fde0 con 0x564639049860 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd99bfff700 1 -- 172.21.15.83:0/1279992687 <== mon.1 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564638f3f190 con 0x564639049860 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd99bfff700 1 -- 172.21.15.83:0/1279992687 <== mon.1 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd9bc006ce0 con 0x564639049860 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 10 client.5572 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 10 client.5572 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 20 client.5572 populate_metadata read hostname 'smithi083' 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x56463906c660 0x564639041340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 4 client.5572 mds_command: new command op to 5763 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564638df5f60 con 0x56463906c660 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.942+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x56463906c660 0x564639041340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd99bfff700 10 client.5572 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x56463906c660 0x564639041340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ebc053cf920266 server_cookie=b534f1ed791d3e37 in_seq=0 out_seq=0 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd99bfff700 1 -- 172.21.15.83:0/1279992687 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x564638df5f60 con 0x56463906c660 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd99bfff700 10 client.5572 handle_command_reply: tid=0 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 client.5572 shutdown 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 msgr2=0x564639067b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 0x564639067b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc003bf0 tx=0x7fd9bc003f00 comp rx=0 tx=0).stop 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 shutdown_connections 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x5646390680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x56463906c660 0x564639041340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1279992687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639049860 0x564639067b80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd99bfff700 0 client.5572 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.943+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 >> 172.21.15.83:0/1279992687 conn(0x56463901cdb0 msgr2=0x56463901e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.944+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 shutdown_connections 2023-11-28T11:10:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.944+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1279992687 wait complete. 2023-11-28T11:10:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.944+0000 7fd9d3937b80 20 client.5572 trim_cache size 0 max 16384 2023-11-28T11:10:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.960+0000 7fd9d3937b80 1 Processor -- start 2023-11-28T11:10:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.960+0000 7fd9d3937b80 1 -- start start 2023-11-28T11:10:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639047180 0x564639066050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x5646390184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x56463901d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564639018fe0 con 0x56463904a170 2023-11-28T11:10:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56463903fae0 con 0x564639047180 2023-11-28T11:10:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x564639041520 con 0x564639049860 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x56463901d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x56463901d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51982/0 (socket says 172.21.15.83:51982) 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/1862884987 learned_addr learned my addr 172.21.15.83:0/1862884987 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639047180 0x564639066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x5646390184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 msgr2=0x5646390184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x5646390184a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639047180 msgr2=0x564639066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639047180 0x564639066050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 -- 172.21.15.83:0/1862884987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564638f38190 con 0x56463904a170 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c554a700 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639047180 0x564639066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x56463901d460 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8054680 tx=0x7fd9b806c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee82043a79c2ef32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9a37fe700 1 -- 172.21.15.83:0/1862884987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8083040 con 0x56463904a170 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9a37fe700 1 -- 172.21.15.83:0/1862884987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8084240 con 0x56463904a170 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.961+0000 7fd9a37fe700 1 -- 172.21.15.83:0/1862884987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8052670 con 0x56463904a170 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1862884987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 msgr2=0x56463901d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x56463901d460 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8054680 tx=0x7fd9b806c040 comp rx=0 tx=0).stop 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1862884987 shutdown_connections 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56463904a170 0x56463901d460 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564639047180 0x564639066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/1862884987 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639049860 0x5646390184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1862884987 >> 172.21.15.83:0/1862884987 conn(0x56463901cdb0 msgr2=0x5646390402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1862884987 shutdown_connections 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.962+0000 7fd9d3937b80 1 -- 172.21.15.83:0/1862884987 wait complete. 2023-11-28T11:10:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.963+0000 7fd9d3937b80 1 -- start start 2023-11-28T11:10:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.963+0000 7fd9d3937b80 1 Processor -- start 2023-11-28T11:10:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.963+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.963+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 0x564639066430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.963+0000 7fd9d3937b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x564639042910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.963+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564639066970 con 0x564639049860 2023-11-28T11:10:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x564639043590 con 0x56463904a170 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9d3937b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56463901e950 con 0x564639047180 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9d3937b80 10 client.0 _ll_register_callbacks cb 0x564638e17290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 0x564639066430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 0x564639066430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51986/0 (socket says 172.21.15.83:51986) 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 -- 172.21.15.83:0/2750482789 learned_addr learned my addr 172.21.15.83:0/2750482789 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c554a700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c5d4b700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x564639042910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 -- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 msgr2=0x5646390cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 -- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 msgr2=0x564639042910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x564639042910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 -- 172.21.15.83:0/2750482789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564638df3630 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9c4d49700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 0x564639066430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc007850 tx=0x7fd9bc007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6997b16fa5a70d49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd99bfff700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd99bfff700 1 -- 172.21.15.83:0/2750482789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc02c070 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd99bfff700 1 -- 172.21.15.83:0/2750482789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd9bc001bb0 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564638f2f1b0 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd99bfff700 1 -- 172.21.15.83:0/2750482789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9bc0070f0 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9d3937b80 10 client.5994 fetch_fsmap learned FSMap version 25 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564638dc6250 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.964+0000 7fd99bfff700 1 -- 172.21.15.83:0/2750482789 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564638f2f1b0 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd99bfff700 1 -- 172.21.15.83:0/2750482789 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd9bc0074e0 con 0x564639049860 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 10 client.5994 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 10 client.5994 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 20 client.5994 populate_metadata read hostname 'smithi083' 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x564639049c30 0x56463901d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 4 client.5994 mds_command: new command op to 5763 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564638f191c0 con 0x564639049c30 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9c554a700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x564639049c30 0x56463901d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd99bfff700 10 client.5994 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9c554a700 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x564639049c30 0x56463901d180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1478ca83bcf3a0 server_cookie=38fd938e3bcf5073 in_seq=0 out_seq=0 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd99bfff700 1 -- 172.21.15.83:0/2750482789 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x564638f191c0 con 0x564639049c30 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd99bfff700 10 client.5994 handle_command_reply: tid=0 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 client.5994 shutdown 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 msgr2=0x564639066430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 0x564639066430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc007850 tx=0x7fd9bc007d50 comp rx=0 tx=0).stop 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 shutdown_connections 2023-11-28T11:10:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x564639049c30 0x56463901d180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56463904a170 0x564639042910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd99bfff700 0 client.5994 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x564639047180 0x5646390cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 --2- 172.21.15.83:0/2750482789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564639049860 0x564639066430 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 >> 172.21.15.83:0/2750482789 conn(0x56463901cdb0 msgr2=0x5646390186c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 shutdown_connections 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.965+0000 7fd9d3937b80 1 -- 172.21.15.83:0/2750482789 wait complete. 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9d3937b80 20 client.5994 trim_cache size 0 max 16384 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:10:34.974 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: "261043a7-cd1e-4c3b-b658-0c6433ba5db3": { 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "261043a7-cd1e-4c3b-b658-0c6433ba5db3", 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:10:34.975 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd9a40682f0 msgr2=0x7fd9a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd9a40682f0 0x7fd9a406a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c240 tx=0x7fd9b000a000 comp rx=0 tx=0).stop 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 msgr2=0x7fd9c015f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 0x7fd9c015f9f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0070610 tx=0x7fd9b800f490 comp rx=0 tx=0).stop 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 shutdown_connections 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd9a40682f0 0x7fd9a406a7a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9c015f580 0x7fd9c015f9f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd9ac003bc0 0x7fd9c015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 --2- 172.21.15.83:0/378242391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9c0149f50 0x7fd9c015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 >> 172.21.15.83:0/378242391 conn(0x7fd9c00b2b60 msgr2=0x7fd9c015c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 shutdown_connections 2023-11-28T11:10:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:34.966+0000 7fd9c77ae700 1 -- 172.21.15.83:0/378242391 wait complete. 2023-11-28T11:10:34.979 INFO:tasks.cephfs.filesystem:scrub status for tag:261043a7-cd1e-4c3b-b658-0c6433ba5db3 - {'path': '/', 'tag': '261043a7-cd1e-4c3b-b658-0c6433ba5db3', 'options': 'recursive,repair'} 2023-11-28T11:10:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:35.419+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-28T11:10:34.785629+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f8e8405f0d0 con 0x7f8e88070430 2023-11-28T11:10:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: cluster: 2023-11-28T11:10:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: id: 69424f07-94c3-4dbc-8507-3551775e0a09 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: health: HEALTH_OK 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: services: 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: mgr: y(active, since 2m), standbys: x 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: data: 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: volumes: 1/1 healthy 2023-11-28T11:10:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: pools: 4 pools, 105 pgs 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: objects: 23 objects, 579 KiB 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: usage: 74 MiB used, 720 GiB / 720 GiB avail 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: pgs: 105 active+clean 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: io: 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: client: 1.8 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:34.785629+0000 mds.d [INF] scrub queued for path: / 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:34.785634+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:34.785641+0000 mds.d [INF] scrub summary: active paths [/] 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:34.786450+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T11:10:35.423 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:34.786458+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T11:10:35.424 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:34.786892+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:10:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:10:37.426+0000 7f8e7effd700 1 -- 172.21.15.83:0/4291462369 <== mon.0 v2:172.21.15.83:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-28T11:10:36.026309+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8e84087080 con 0x7f8e88070430 2023-11-28T11:10:37.427 INFO:teuthology.orchestra.run.smithi083.stdout:2023-11-28T11:10:36.026309+0000 mds.d [INF] scrub summary: idle 2023-11-28T11:10:39.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:39.580 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:04.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:11:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.114+0000 7fe858f20700 1 Processor -- start 2023-11-28T11:11:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.114+0000 7fe858f20700 1 -- start start 2023-11-28T11:11:05.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.115+0000 7fe858f20700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe8540cebb0 0x7fe8541598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.115+0000 7fe858f20700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe85415d590 0x7fe85415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.115+0000 7fe858f20700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85415d100 con 0x7fe85406e920 2023-11-28T11:11:05.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.115+0000 7fe858f20700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe854159df0 con 0x7fe85415d590 2023-11-28T11:11:05.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.115+0000 7fe858f20700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8540cf030 con 0x7fe8540cebb0 2023-11-28T11:11:05.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe85415d590 0x7fe85415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe85415d590 0x7fe85415d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51996/0 (socket says 172.21.15.83:51996) 2023-11-28T11:11:05.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 -- 172.21.15.83:0/1807764142 learned_addr learned my addr 172.21.15.83:0/1807764142 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe851d9b700 1 --2- 172.21.15.83:0/1807764142 >> v2:172.21.15.90:3300/0 conn(0x7fe8540cebb0 0x7fe8541598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 -- 172.21.15.83:0/1807764142 >> v2:172.21.15.90:3300/0 conn(0x7fe8540cebb0 msgr2=0x7fe8541598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 --2- 172.21.15.83:0/1807764142 >> v2:172.21.15.90:3300/0 conn(0x7fe8540cebb0 0x7fe8541598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 -- 172.21.15.83:0/1807764142 >> v1:172.21.15.90:6789/0 conn(0x7fe85406e920 legacy=0x7fe85406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 -- 172.21.15.83:0/1807764142 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85415a050 con 0x7fe85415d590 2023-11-28T11:11:05.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe851d9b700 1 --2- 172.21.15.83:0/1807764142 >> v2:172.21.15.90:3300/0 conn(0x7fe8540cebb0 0x7fe8541598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe852d9d700 1 --2- 172.21.15.83:0/1807764142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe85415d9d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00d150 tx=0x7fe84c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97a4f3513e6c5f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe85159a700 1 -- 172.21.15.83:0/1807764142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c010070 con 0x7fe85415d590 2023-11-28T11:11:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe85159a700 1 -- 172.21.15.83:0/1807764142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00dd40 con 0x7fe85415d590 2023-11-28T11:11:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.116+0000 7fe85159a700 1 -- 172.21.15.83:0/1807764142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00b310 con 0x7fe85415d590 2023-11-28T11:11:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 -- 172.21.15.83:0/1807764142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 msgr2=0x7fe85415d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 --2- 172.21.15.83:0/1807764142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe85415d9d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00d150 tx=0x7fe84c00d460 comp rx=0 tx=0).stop 2023-11-28T11:11:05.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 -- 172.21.15.83:0/1807764142 shutdown_connections 2023-11-28T11:11:05.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 --2- 172.21.15.83:0/1807764142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe85415d9d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 --2- 172.21.15.83:0/1807764142 >> v2:172.21.15.90:3300/0 conn(0x7fe8540cebb0 0x7fe8541598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 -- 172.21.15.83:0/1807764142 >> 172.21.15.83:0/1807764142 conn(0x7fe8540cf300 msgr2=0x7fe8540cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 -- 172.21.15.83:0/1807764142 shutdown_connections 2023-11-28T11:11:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.117+0000 7fe858f20700 1 -- 172.21.15.83:0/1807764142 wait complete. 2023-11-28T11:11:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.118+0000 7fe858f20700 1 Processor -- start 2023-11-28T11:11:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.118+0000 7fe858f20700 1 -- start start 2023-11-28T11:11:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe858f20700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe85406e920 0x7fe8540c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe858f20700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8540cebb0 0x7fe8541f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe858f20700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe854155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe858f20700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8541f20f0 con 0x7fe85415d590 2023-11-28T11:11:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe858f20700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c9240 con 0x7fe85406e920 2023-11-28T11:11:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe858f20700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c7920 con 0x7fe8540cebb0 2023-11-28T11:11:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe854155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe854155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52000/0 (socket says 172.21.15.83:52000) 2023-11-28T11:11:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 -- 172.21.15.83:0/3558121112 learned_addr learned my addr 172.21.15.83:0/3558121112 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe85259c700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe85406e920 0x7fe8540c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe851d9b700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8540cebb0 0x7fe8541f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 -- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8540cebb0 msgr2=0x7fe8541f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8540cebb0 0x7fe8541f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 -- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe85406e920 msgr2=0x7fe8540c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe85406e920 0x7fe8540c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe852d9d700 1 -- 172.21.15.83:0/3558121112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c00bd40 con 0x7fe85415d590 2023-11-28T11:11:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.119+0000 7fe851d9b700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8540cebb0 0x7fe8541f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.120+0000 7fe852d9d700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe854155620 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe854071000 tx=0x7fe84c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.120+0000 7fe84affd700 1 -- 172.21.15.83:0/3558121112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00ab20 con 0x7fe85415d590 2023-11-28T11:11:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.120+0000 7fe84affd700 1 -- 172.21.15.83:0/3558121112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00ac80 con 0x7fe85415d590 2023-11-28T11:11:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.120+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85415a610 con 0x7fe85415d590 2023-11-28T11:11:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.120+0000 7fe84affd700 1 -- 172.21.15.83:0/3558121112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c004030 con 0x7fe85415d590 2023-11-28T11:11:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.120+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85415a330 con 0x7fe85415d590 2023-11-28T11:11:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.121+0000 7fe84affd700 1 -- 172.21.15.83:0/3558121112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe84c004190 con 0x7fe85415d590 2023-11-28T11:11:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.122+0000 7fe84affd700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8340682f0 0x7fe83406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.122+0000 7fe84affd700 1 -- 172.21.15.83:0/3558121112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe84c089af0 con 0x7fe85415d590 2023-11-28T11:11:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.122+0000 7fe85259c700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8340682f0 0x7fe83406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.122+0000 7fe85259c700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8340682f0 0x7fe83406a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe844004f30 tx=0x7fe844009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.123+0000 7fe8650a9b80 1 Processor -- start 2023-11-28T11:11:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 -- start start 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aaddbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddc170 0x5645aadf77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aac9bc10 con 0x5645aaddb860 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aad0a3a0 con 0x5645aadd9180 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645aad038d0 con 0x5645aaddc170 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe85259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe85259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52006/0 (socket says 172.21.15.83:52006) 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.124+0000 7fe85259c700 1 -- 172.21.15.83:0/2553442622 learned_addr learned my addr 172.21.15.83:0/2553442622 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe85259c700 1 -- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddc170 msgr2=0x5645aadf77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe85259c700 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddc170 0x5645aadf77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe85259c700 1 -- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 msgr2=0x5645aaddbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe851d9b700 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aaddbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe85259c700 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aaddbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe85259c700 1 -- 172.21.15.83:0/2553442622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645aab85630 con 0x5645aaddb860 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe85259c700 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1a70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe844015720 tx=0x7fe844011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cab64aafe100015 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe833fff700 1 -- 172.21.15.83:0/2553442622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440135f0 con 0x5645aaddb860 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe833fff700 1 -- 172.21.15.83:0/2553442622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe844013750 con 0x5645aaddb860 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.125+0000 7fe833fff700 1 -- 172.21.15.83:0/2553442622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844013ac0 con 0x5645aaddb860 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 -- 172.21.15.83:0/2553442622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 msgr2=0x5645aadb1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1a70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe844015720 tx=0x7fe844011000 comp rx=0 tx=0).stop 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 -- 172.21.15.83:0/2553442622 shutdown_connections 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddc170 0x5645aadf77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1a70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/2553442622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aaddbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 -- 172.21.15.83:0/2553442622 >> 172.21.15.83:0/2553442622 conn(0x5645aadaedb0 msgr2=0x5645aadaf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 -- 172.21.15.83:0/2553442622 shutdown_connections 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 -- 172.21.15.83:0/2553442622 wait complete. 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 -- start start 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.126+0000 7fe8650a9b80 1 Processor -- start 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aadca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadcaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 0x5645aadcaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aab87920 con 0x5645aaddc170 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aadedcd0 con 0x5645aadd9180 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645aadcea90 con 0x5645aaddb860 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe8650a9b80 10 client.0 _ll_register_callbacks cb 0x5645aaba9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.127+0000 7fe852d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 0x5645aadcaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 0x5645aadcaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52016/0 (socket says 172.21.15.83:52016) 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 -- 172.21.15.83:0/613650963 learned_addr learned my addr 172.21.15.83:0/613650963 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe85259c700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aadca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe851d9b700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadcaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 -- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 msgr2=0x5645aadcaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadcaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 -- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 msgr2=0x5645aadca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aadca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 -- 172.21.15.83:0/613650963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84400f040 con 0x5645aaddc170 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe852d9d700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 0x5645aadcaf80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe84c059db0 tx=0x7fe84c0596a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49afbb7139b853d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8317fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8317fa700 1 -- 172.21.15.83:0/613650963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c058e30 con 0x5645aaddc170 2023-11-28T11:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8317fa700 1 -- 172.21.15.83:0/613650963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe84c058f90 con 0x5645aaddc170 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645aacd1190 con 0x5645aaddc170 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8317fa700 1 -- 172.21.15.83:0/613650963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe84c059150 con 0x5645aaddc170 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 10 client.6006 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5645aab31de0 con 0x5645aaddc170 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8317fa700 1 -- 172.21.15.83:0/613650963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645aacd1190 con 0x5645aaddc170 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8317fa700 1 -- 172.21.15.83:0/613650963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe84c05abe0 con 0x5645aaddc170 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 10 client.6006 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 10 client.6006 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 20 client.6006 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aaded5d0 0x5645aadda9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 4 client.6006 mds_command: new command op to 5763 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.128+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645aab87f60 con 0x5645aaded5d0 2023-11-28T11:11:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.129+0000 7fe85259c700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aaded5d0 0x5645aadda9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.129+0000 7fe85259c700 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aaded5d0 0x5645aadda9f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529d1851c4dc590b server_cookie=f014dc8312e28f44 in_seq=0 out_seq=0 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.129+0000 7fe8317fa700 10 client.6006 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8317fa700 1 -- 172.21.15.83:0/613650963 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5645aab87f60 con 0x5645aaded5d0 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8317fa700 10 client.6006 handle_command_reply: tid=0 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 client.6006 shutdown 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 msgr2=0x5645aadcaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 0x5645aadcaf80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe84c059db0 tx=0x7fe84c0596a0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 shutdown_connections 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddc170 0x5645aadcaf80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aaded5d0 0x5645aadda9f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aadd9180 0x5645aadca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8317fa700 0 client.6006 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:11:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/613650963 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadcaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 >> 172.21.15.83:0/613650963 conn(0x5645aadaedb0 msgr2=0x5645aadb19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 shutdown_connections 2023-11-28T11:11:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 1 -- 172.21.15.83:0/613650963 wait complete. 2023-11-28T11:11:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.130+0000 7fe8650a9b80 20 client.6006 trim_cache size 0 max 16384 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.146+0000 7fe8650a9b80 1 Processor -- start 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.146+0000 7fe8650a9b80 1 -- start start 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aadd9180 0x5645aadb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadd85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aadab980 con 0x5645aaddb860 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aadd1e00 con 0x5645aaddc170 2023-11-28T11:11:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645aadb2f00 con 0x5645aadd9180 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52022/0 (socket says 172.21.15.83:52022) 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 -- 172.21.15.83:0/1792892877 learned_addr learned my addr 172.21.15.83:0/1792892877 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe852d9d700 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadd85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe85259c700 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aadd9180 0x5645aadb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 -- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aadd9180 msgr2=0x5645aadb1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aadd9180 0x5645aadb1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 -- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 msgr2=0x5645aadd85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadd85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe851d9b700 1 -- 172.21.15.83:0/1792892877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645aacca190 con 0x5645aaddb860 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.147+0000 7fe852d9d700 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadd85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe851d9b700 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1b40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00da60 tx=0x7fe83c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec00960884f57c2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe848ff9700 1 -- 172.21.15.83:0/1792892877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00fba0 con 0x5645aaddb860 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe848ff9700 1 -- 172.21.15.83:0/1792892877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c00fd20 con 0x5645aaddb860 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe848ff9700 1 -- 172.21.15.83:0/1792892877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c012660 con 0x5645aaddb860 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 -- 172.21.15.83:0/1792892877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 msgr2=0x5645aadb1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1b40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00da60 tx=0x7fe83c00dd70 comp rx=0 tx=0).stop 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 -- 172.21.15.83:0/1792892877 shutdown_connections 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadd85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aadd9180 0x5645aadb1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/1792892877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aaddb860 0x5645aadb1b40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 -- 172.21.15.83:0/1792892877 >> 172.21.15.83:0/1792892877 conn(0x5645aadaedb0 msgr2=0x5645aadad150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 -- 172.21.15.83:0/1792892877 shutdown_connections 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.148+0000 7fe8650a9b80 1 -- 172.21.15.83:0/1792892877 wait complete. 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.149+0000 7fe8650a9b80 1 -- start start 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.149+0000 7fe8650a9b80 1 Processor -- start 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 0x5645aadf9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadd1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadaf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aadee020 con 0x5645aadd9180 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645aadd2cd0 con 0x5645aaddc170 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645aae0f170 con 0x5645aaddb860 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8650a9b80 10 client.0 _ll_register_callbacks cb 0x5645aaba9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 0x5645aadf9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 0x5645aadf9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52026/0 (socket says 172.21.15.83:52026) 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 -- 172.21.15.83:0/3991404782 learned_addr learned my addr 172.21.15.83:0/3991404782 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe852d9d700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadaf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe851d9b700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadd1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 -- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 msgr2=0x5645aadd1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadd1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 -- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 msgr2=0x5645aadaf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadaf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 -- 172.21.15.83:0/3991404782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c00d690 con 0x5645aadd9180 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe85259c700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 0x5645aadf9ba0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe844007fd0 tx=0x7fe844003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f2772e3914b965 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8317fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:05.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8317fa700 1 -- 172.21.15.83:0/3991404782 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844013ac0 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.150+0000 7fe8317fa700 1 -- 172.21.15.83:0/3991404782 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe844013c20 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645aacc11b0 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8317fa700 1 -- 172.21.15.83:0/3991404782 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe844013de0 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 10 client.6012 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5645aab58250 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8317fa700 1 -- 172.21.15.83:0/3991404782 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645aacc11b0 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8317fa700 1 -- 172.21.15.83:0/3991404782 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe84402c450 con 0x5645aadd9180 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 10 client.6012 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 10 client.6012 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 20 client.6012 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aadcb690 0x5645aadaf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 4 client.6012 mds_command: new command op to 5763 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 --> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5645aacab1c0 con 0x5645aadcb690 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe851d9b700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aadcb690 0x5645aadaf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe8317fa700 10 client.6012 ms_handle_connect on v2:172.21.15.90:6833/3041933349 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.151+0000 7fe851d9b700 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aadcb690 0x5645aadaf6c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c98102dd43cf8c0 server_cookie=8c46662412a04dd6 in_seq=0 out_seq=0 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8317fa700 1 -- 172.21.15.83:0/3991404782 <== mds.0 v2:172.21.15.90:6833/3041933349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5645aacab1c0 con 0x5645aadcb690 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8317fa700 10 client.6012 handle_command_reply: tid=0 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 client.6012 shutdown 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 msgr2=0x5645aadf9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 0x5645aadf9ba0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe844007fd0 tx=0x7fe844003780 comp rx=0 tx=0).stop 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 shutdown_connections 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:6833/3041933349,v1:172.21.15.90:6835/3041933349] conn(0x5645aadcb690 0x5645aadaf6c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645aaddc170 0x5645aadaf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645aadd9180 0x5645aadf9ba0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 --2- 172.21.15.83:0/3991404782 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5645aaddb860 0x5645aadd1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8317fa700 0 client.6012 ms_handle_reset on v2:172.21.15.90:6833/3041933349 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 >> 172.21.15.83:0/3991404782 conn(0x5645aadaedb0 msgr2=0x5645aadc92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 shutdown_connections 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 1 -- 172.21.15.83:0/3991404782 wait complete. 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe8650a9b80 20 client.6012 trim_cache size 0 max 16384 2023-11-28T11:11:05.162 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:05.163 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2023-11-28T11:11:05.163 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2023-11-28T11:11:05.163 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8340682f0 msgr2=0x7fe83406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe858f20700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8340682f0 0x7fe83406a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe844004f30 tx=0x7fe844009040 comp rx=0 tx=0).stop 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 msgr2=0x7fe854155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.152+0000 7fe858f20700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe854155620 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe854071000 tx=0x7fe84c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 shutdown_connections 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8340682f0 0x7fe83406a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe85415d590 0x7fe854155620 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe85406e920 0x7fe8540c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 --2- 172.21.15.83:0/3558121112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8540cebb0 0x7fe8541f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 >> 172.21.15.83:0/3558121112 conn(0x7fe8540cf300 msgr2=0x7fe8540cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 shutdown_connections 2023-11-28T11:11:05.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.153+0000 7fe858f20700 1 -- 172.21.15.83:0/3558121112 wait complete. 2023-11-28T11:11:05.167 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:11:05.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:05.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.387+0000 7fcda69a2700 1 Processor -- start 2023-11-28T11:11:05.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.387+0000 7fcda69a2700 1 -- start start 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.387+0000 7fcda69a2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda015a480 0x7fcda00cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda69a2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcda00cfce0 0x7fcda00b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda69a2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda015efa0 con 0x7fcda015a850 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda69a2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00ceab0 con 0x7fcda00cfce0 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda69a2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00cecc0 con 0x7fcda015a480 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda4f3f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcda00cfce0 0x7fcda00b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcda015a850 0x7fcda00b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49896/0 (socket says 172.21.15.83:49896) 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9f7fe700 1 -- 172.21.15.83:0/3232245943 learned_addr learned my addr 172.21.15.83:0/3232245943 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9ffff700 1 --2- 172.21.15.83:0/3232245943 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda015a480 0x7fcda00cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda4f3f700 1 -- 172.21.15.83:0/3232245943 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda015a480 msgr2=0x7fcda00cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda4f3f700 1 --2- 172.21.15.83:0/3232245943 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda015a480 0x7fcda00cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda4f3f700 1 -- 172.21.15.83:0/3232245943 >> v1:172.21.15.83:6789/0 conn(0x7fcda015a850 legacy=0x7fcda00b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9effd700 1 -- 172.21.15.83:0/3232245943 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821655204 0 0) 0x7fcda015efa0 con 0x7fcda015a850 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda4f3f700 1 -- 172.21.15.83:0/3232245943 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00cefa0 con 0x7fcda00cfce0 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcda4f3f700 1 --2- 172.21.15.83:0/3232245943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda00cfce0 0x7fcda00b2960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009d90 tx=0x7fcd9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607772f08259f637 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9effd700 1 -- 172.21.15.83:0/3232245943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800e070 con 0x7fcda00cfce0 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9effd700 1 -- 172.21.15.83:0/3232245943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9800bbd0 con 0x7fcda00cfce0 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.388+0000 7fcd9effd700 1 -- 172.21.15.83:0/3232245943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98012430 con 0x7fcda00cfce0 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 -- 172.21.15.83:0/3232245943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda00cfce0 msgr2=0x7fcda00b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 --2- 172.21.15.83:0/3232245943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda00cfce0 0x7fcda00b2960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009d90 tx=0x7fcd9800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 -- 172.21.15.83:0/3232245943 shutdown_connections 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 --2- 172.21.15.83:0/3232245943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda00cfce0 0x7fcda00b2960 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 --2- 172.21.15.83:0/3232245943 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda015a480 0x7fcda00cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 -- 172.21.15.83:0/3232245943 >> 172.21.15.83:0/3232245943 conn(0x7fcda006de20 msgr2=0x7fcda006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 -- 172.21.15.83:0/3232245943 shutdown_connections 2023-11-28T11:11:05.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.389+0000 7fcda69a2700 1 -- 172.21.15.83:0/3232245943 wait complete. 2023-11-28T11:11:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.390+0000 7fcda69a2700 1 Processor -- start 2023-11-28T11:11:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.390+0000 7fcda69a2700 1 -- start start 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda69a2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda00cfce0 0x7fcda0153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda69a2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda015a480 0x7fcda00ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda69a2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 0x7fcda0155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda69a2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda01efea0 con 0x7fcda015a850 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda69a2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0153b70 con 0x7fcda015a480 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda69a2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00cacf0 con 0x7fcda00cfce0 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 0x7fcda0155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 0x7fcda0155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52044/0 (socket says 172.21.15.83:52044) 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 -- 172.21.15.83:0/1423552659 learned_addr learned my addr 172.21.15.83:0/1423552659 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcd9f7fe700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda015a480 0x7fcda00ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcd9ffff700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda00cfce0 0x7fcda0153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 -- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda00cfce0 msgr2=0x7fcda0153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda00cfce0 0x7fcda0153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 -- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda015a480 msgr2=0x7fcda00ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda015a480 0x7fcda00ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcda4f3f700 1 -- 172.21.15.83:0/1423552659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd98009a70 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.391+0000 7fcd9ffff700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda00cfce0 0x7fcda0153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.392+0000 7fcda4f3f700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 0x7fcda0155810 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcda00b3ad0 tx=0x7fcd98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.392+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98012430 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.392+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98020070 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.392+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda00b4440 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.392+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98004280 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.392+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda00cf280 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.393+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda00b45d0 con 0x7fcda015a850 2023-11-28T11:11:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.393+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd980043e0 con 0x7fcda015a850 2023-11-28T11:11:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.394+0000 7fcd9cff9700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcd8c0682f0 0x7fcd8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.394+0000 7fcd9ffff700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcd8c0682f0 0x7fcd8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.394+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcd98022070 con 0x7fcda015a850 2023-11-28T11:11:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.394+0000 7fcd9ffff700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcd8c0682f0 0x7fcd8c06a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcda015c980 tx=0x7fcd90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.398+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd98054c70 con 0x7fcda015a850 2023-11-28T11:11:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.623+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcda0074320 con 0x7fcda015a850 2023-11-28T11:11:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.623+0000 7fcd9cff9700 1 -- 172.21.15.83:0/1423552659 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fcd98025380 con 0x7fcda015a850 2023-11-28T11:11:05.625 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:05.625 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcd8c0682f0 msgr2=0x7fcd8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcd8c0682f0 0x7fcd8c06a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcda015c980 tx=0x7fcd90008040 comp rx=0 tx=0).stop 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 msgr2=0x7fcda0155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 0x7fcda0155810 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcda00b3ad0 tx=0x7fcd98004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 shutdown_connections 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcd8c0682f0 0x7fcd8c06a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcda015a850 0x7fcda0155810 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcda00cfce0 0x7fcda0153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 --2- 172.21.15.83:0/1423552659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda015a480 0x7fcda00ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 >> 172.21.15.83:0/1423552659 conn(0x7fcda006de20 msgr2=0x7fcda00cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 shutdown_connections 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.625+0000 7fcda69a2700 1 -- 172.21.15.83:0/1423552659 wait complete. 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:11:05.639 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:11:05.795 INFO:teuthology.orchestra.run.smithi090.stdout:{"message":"","return_code":0} 2023-11-28T11:11:05.796 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:11:05.796 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:11:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.838+0000 7f834d5ee080 1 Processor -- start 2023-11-28T11:11:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.838+0000 7f834d5ee080 1 -- start start 2023-11-28T11:11:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f834d5ee080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 0x557716f47280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f834d5ee080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f4d170 0x557716f515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f834d5ee080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716f3fc00 0x557716ea4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f834d5ee080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716daffe0 con 0x557716e75e70 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f834d5ee080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716ea39e0 con 0x557716f3fc00 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f834d5ee080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x557716f3f690 con 0x557716f4d170 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f83436f1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 0x557716f47280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f83436f1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 0x557716f47280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52054/0 (socket says 172.21.15.83:52054) 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f83436f1700 1 -- 172.21.15.83:0/4116769447 learned_addr learned my addr 172.21.15.83:0/4116769447 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.839+0000 7f8342ef0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f4d170 0x557716f515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f8343ef2700 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716f3fc00 0x557716ea4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f83436f1700 1 -- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f4d170 msgr2=0x557716f515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f83436f1700 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f4d170 0x557716f515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f83436f1700 1 -- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716f3fc00 msgr2=0x557716ea4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f83436f1700 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716f3fc00 0x557716ea4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f83436f1700 1 -- 172.21.15.83:0/4116769447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c00a1c0 con 0x557716e75e70 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f8343ef2700 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716f3fc00 0x557716ea4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f83436f1700 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 0x557716f47280 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8334001d40 tx=0x7f833400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5461296a1f0e67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f8341eee700 1 -- 172.21.15.83:0/4116769447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334014070 con 0x557716e75e70 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f8341eee700 1 -- 172.21.15.83:0/4116769447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334001e90 con 0x557716e75e70 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f8341eee700 1 -- 172.21.15.83:0/4116769447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400bc60 con 0x557716e75e70 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 -- 172.21.15.83:0/4116769447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 msgr2=0x557716f47280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 0x557716f47280 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8334001d40 tx=0x7f833400b4c0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 -- 172.21.15.83:0/4116769447 shutdown_connections 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716f3fc00 0x557716ea4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716e75e70 0x557716f47280 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 --2- 172.21.15.83:0/4116769447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f4d170 0x557716f515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 -- 172.21.15.83:0/4116769447 >> 172.21.15.83:0/4116769447 conn(0x557716ec0af0 msgr2=0x557716f48d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 -- 172.21.15.83:0/4116769447 shutdown_connections 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.840+0000 7f834d5ee080 1 -- 172.21.15.83:0/4116769447 wait complete. 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.841+0000 7f834d5ee080 1 Processor -- start 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.841+0000 7f834d5ee080 1 -- start start 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f834d5ee080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716e75e70 0x557716f3eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f834d5ee080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f3fc00 0x557716ea4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f834d5ee080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716f4d170 0x557716f4b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f834d5ee080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716ea52c0 con 0x557716f4d170 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f834d5ee080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716ebafd0 con 0x557716e75e70 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f834d5ee080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x557716ea4fe0 con 0x557716f3fc00 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716f4d170 0x557716f4b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f83436f1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716e75e70 0x557716f3eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8342ef0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f3fc00 0x557716ea4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716f4d170 0x557716f4b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52062/0 (socket says 172.21.15.83:52062) 2023-11-28T11:11:05.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 -- 172.21.15.83:0/244526469 learned_addr learned my addr 172.21.15.83:0/244526469 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 -- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f3fc00 msgr2=0x557716ea4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x557716f3fc00 0x557716ea4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 -- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716e75e70 msgr2=0x557716f3eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557716e75e70 0x557716f3eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 -- 172.21.15.83:0/244526469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833400a040 con 0x557716f4d170 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.842+0000 7f8343ef2700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716f4d170 0x557716f4b870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f833c00a880 tx=0x7f833c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.843+0000 7f8340eec700 1 -- 172.21.15.83:0/244526469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c010c00 con 0x557716f4d170 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.843+0000 7f8340eec700 1 -- 172.21.15.83:0/244526469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c009040 con 0x557716f4d170 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.843+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557716dc8120 con 0x557716f4d170 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.843+0000 7f8340eec700 1 -- 172.21.15.83:0/244526469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00ee90 con 0x557716f4d170 2023-11-28T11:11:05.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.843+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557716dbb830 con 0x557716f4d170 2023-11-28T11:11:05.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.844+0000 7f8340eec700 1 -- 172.21.15.83:0/244526469 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f833c00e070 con 0x557716f4d170 2023-11-28T11:11:05.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.844+0000 7f8340eec700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8330066c10 0x7f83300690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.844+0000 7f8340eec700 1 -- 172.21.15.83:0/244526469 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f833c0869f0 con 0x557716f4d170 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.844+0000 7f83436f1700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8330066c10 0x7f83300690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x557716e5fd60 0x557716fe6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x557716f48d10 con 0x557716e5fd60 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f83436f1700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8330066c10 0x7f83300690c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8334001d40 tx=0x7f8334009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f8342ef0700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x557716e5fd60 0x557716fe6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f8342ef0700 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x557716e5fd60 0x557716fe6f80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f8342ef0700 1 -- 172.21.15.83:0/244526469 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f8338009aa0 con 0x557716e5fd60 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x557716e5fd60 msgr2=0x557716fe6f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x557716e5fd60 0x557716fe6f80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8330066c10 msgr2=0x7f83300690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f8330066c10 0x7f83300690c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8334001d40 tx=0x7f8334009000 comp rx=0 tx=0).stop 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716f4d170 msgr2=0x557716f4b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.845+0000 7f834d5ee080 1 --2- 172.21.15.83:0/244526469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557716f4d170 0x557716f4b870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f833c00a880 tx=0x7f833c003330 comp rx=0 tx=0).stop 2023-11-28T11:11:05.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.846+0000 7f8343ef2700 1 -- 172.21.15.83:0/244526469 reap_dead start 2023-11-28T11:11:05.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.846+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 shutdown_connections 2023-11-28T11:11:05.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.846+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 >> 172.21.15.83:0/244526469 conn(0x557716ec0af0 msgr2=0x557716f515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.846+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 shutdown_connections 2023-11-28T11:11:05.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:05.846+0000 7f834d5ee080 1 -- 172.21.15.83:0/244526469 wait complete. 2023-11-28T11:11:05.853 INFO:teuthology.orchestra.run.smithi083.stdout:Yalphaparent_a 2023-11-28T11:11:05.853 DEBUG:teuthology.orchestra.run.smithi083:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:11:05.948 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "ancestors": [ 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1099511627776, 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "alpha", 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 2 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1, 2023-11-28T11:11:05.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "parent_a", 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 4 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 9, 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout: "old_pools": [] 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:05.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:05.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:11:05.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:06.038 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:06.078 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:11:06.078 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:11:06.078 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:06.095 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:11:06.096 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:06.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:06.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.311+0000 7f48e1db0700 1 Processor -- start 2023-11-28T11:11:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.311+0000 7f48e1db0700 1 -- start start 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48e1db0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f48dc161020 0x7f48dc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48e1db0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48dc0b1c60 0x7f48dc06b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48e1db0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc0b20b0 con 0x7f48dc0af030 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48e1db0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc0cb820 con 0x7f48dc0b1c60 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48e1db0700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc0ce620 con 0x7f48dc161020 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48dc0b1c60 0x7f48dc06b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48dc0b1c60 0x7f48dc06b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52072/0 (socket says 172.21.15.83:52072) 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 -- 172.21.15.83:0/3526333472 learned_addr learned my addr 172.21.15.83:0/3526333472 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48daffd700 1 --2- 172.21.15.83:0/3526333472 >> v2:172.21.15.90:3300/0 conn(0x7f48dc161020 0x7f48dc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 -- 172.21.15.83:0/3526333472 >> v2:172.21.15.90:3300/0 conn(0x7f48dc161020 msgr2=0x7f48dc0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 --2- 172.21.15.83:0/3526333472 >> v2:172.21.15.90:3300/0 conn(0x7f48dc161020 0x7f48dc0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 -- 172.21.15.83:0/3526333472 >> v1:172.21.15.90:6789/0 conn(0x7f48dc0af030 legacy=0x7f48dc0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 -- 172.21.15.83:0/3526333472 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0ce8c0 con 0x7f48dc0b1c60 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.312+0000 7f48dbfff700 1 --2- 172.21.15.83:0/3526333472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b1c60 0x7f48dc06b050 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0cfc40 tx=0x7f48d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eacc606a748ef45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48da7fc700 1 -- 172.21.15.83:0/3526333472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400e070 con 0x7f48dc0b1c60 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48da7fc700 1 -- 172.21.15.83:0/3526333472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400b910 con 0x7f48dc0b1c60 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48da7fc700 1 -- 172.21.15.83:0/3526333472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400bcc0 con 0x7f48dc0b1c60 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 -- 172.21.15.83:0/3526333472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b1c60 msgr2=0x7f48dc06b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 --2- 172.21.15.83:0/3526333472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b1c60 0x7f48dc06b050 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0cfc40 tx=0x7f48d400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 -- 172.21.15.83:0/3526333472 shutdown_connections 2023-11-28T11:11:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 --2- 172.21.15.83:0/3526333472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b1c60 0x7f48dc06b050 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 --2- 172.21.15.83:0/3526333472 >> v2:172.21.15.90:3300/0 conn(0x7f48dc161020 0x7f48dc0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 -- 172.21.15.83:0/3526333472 >> 172.21.15.83:0/3526333472 conn(0x7f48dc0cc720 msgr2=0x7f48dc0ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 -- 172.21.15.83:0/3526333472 shutdown_connections 2023-11-28T11:11:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.313+0000 7f48e1db0700 1 -- 172.21.15.83:0/3526333472 wait complete. 2023-11-28T11:11:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.314+0000 7f48e1db0700 1 Processor -- start 2023-11-28T11:11:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 -- start start 2023-11-28T11:11:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48dc0af030 0x7f48dc0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48dc0b1c60 0x7f48dc0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 0x7f48dc1e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc1e6110 con 0x7f48dc0b5120 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0b55e0 con 0x7f48dc0b1c60 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48e1db0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0b58c0 con 0x7f48dc0af030 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48dbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 0x7f48dc1e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48dbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 0x7f48dc1e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52078/0 (socket says 172.21.15.83:52078) 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.315+0000 7f48dbfff700 1 -- 172.21.15.83:0/2599370720 learned_addr learned my addr 172.21.15.83:0/2599370720 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48daffd700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48dc0b1c60 0x7f48dc0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48db7fe700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48dc0af030 0x7f48dc0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48dbfff700 1 -- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48dc0af030 msgr2=0x7f48dc0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48dbfff700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48dc0af030 0x7f48dc0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48dbfff700 1 -- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48dc0b1c60 msgr2=0x7f48dc0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48dbfff700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48dc0b1c60 0x7f48dc0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48dbfff700 1 -- 172.21.15.83:0/2599370720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009b30 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48daffd700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48dc0b1c60 0x7f48dc0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48dbfff700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 0x7f48dc1e6b60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f48dc06db00 tx=0x7f48d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40099b0 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d40092e0 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc0cee80 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d401c640 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.316+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc0ceba0 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.317+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc0cf160 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.317+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48d4022030 con 0x7f48dc0b5120 2023-11-28T11:11:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.318+0000 7f48c3fff700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48c80682f0 0x7f48c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.318+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f48d40872f0 con 0x7f48dc0b5120 2023-11-28T11:11:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.318+0000 7f48db7fe700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48c80682f0 0x7f48c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.318+0000 7f48db7fe700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48c80682f0 0x7f48c806a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48d0000f60 tx=0x7f48d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.322+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48d4053a10 con 0x7f48dc0b5120 2023-11-28T11:11:06.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.507+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48dc074320 con 0x7f48dc0b5120 2023-11-28T11:11:06.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.508+0000 7f48c3fff700 1 -- 172.21.15.83:0/2599370720 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f48d4057030 con 0x7f48dc0b5120 2023-11-28T11:11:06.510 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:27.384310+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48c80682f0 msgr2=0x7f48c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48c80682f0 0x7f48c806a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48d0000f60 tx=0x7f48d0009040 comp rx=0 tx=0).stop 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 msgr2=0x7f48dc1e6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 0x7f48dc1e6b60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f48dc06db00 tx=0x7f48d4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 shutdown_connections 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f48c80682f0 0x7f48c806a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48dc0b5120 0x7f48dc1e6b60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f48dc0af030 0x7f48dc0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 --2- 172.21.15.83:0/2599370720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48dc0b1c60 0x7f48dc0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 >> 172.21.15.83:0/2599370720 conn(0x7f48dc0cc720 msgr2=0x7f48dc06b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 shutdown_connections 2023-11-28T11:11:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.510+0000 7f48e1db0700 1 -- 172.21.15.83:0/2599370720 wait complete. 2023-11-28T11:11:06.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:24.134118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:25.340097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:11:06.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:11:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.646+0000 7f43a2b41700 1 Processor -- start 2023-11-28T11:11:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.646+0000 7f43a2b41700 1 -- start start 2023-11-28T11:11:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.647+0000 7f43a2b41700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 0x7f439c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.647+0000 7f43a2b41700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f439c0cd1c0 0x7f439c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.647+0000 7f43a2b41700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c0b4bb0 con 0x7f439c0ccb60 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.647+0000 7f43a2b41700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0d0af0 con 0x7f439c0cd1c0 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.647+0000 7f43a2b41700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c15ac30 con 0x7f439c0aefd0 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f43a08dd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 0x7f439c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439bfff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f439c0cd1c0 0x7f439c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f43a08dd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 0x7f439c0d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:36284/0 (socket says 172.21.15.83:36284) 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f43a08dd700 1 -- 172.21.15.83:0/1362990348 learned_addr learned my addr 172.21.15.83:0/1362990348 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439bfff700 1 -- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 msgr2=0x7f439c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439bfff700 1 --2- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 0x7f439c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439bfff700 1 -- 172.21.15.83:0/1362990348 >> v1:172.21.15.90:6789/0 conn(0x7f439c0ccb60 legacy=0x7f439c15e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439bfff700 1 -- 172.21.15.83:0/1362990348 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0ccfa0 con 0x7f439c0cd1c0 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f43a08dd700 1 --2- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 0x7f439c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439bfff700 1 --2- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0cc620 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f4388009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15893b7bf31bfd96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439b7fe700 1 -- 172.21.15.83:0/1362990348 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388004230 con 0x7f439c0cd1c0 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439b7fe700 1 -- 172.21.15.83:0/1362990348 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4388004390 con 0x7f439c0cd1c0 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.648+0000 7f439b7fe700 1 -- 172.21.15.83:0/1362990348 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43880036a0 con 0x7f439c0cd1c0 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 -- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 msgr2=0x7f439c0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 --2- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0cc620 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f4388009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 -- 172.21.15.83:0/1362990348 shutdown_connections 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 --2- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0aefd0 0x7f439c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 --2- 172.21.15.83:0/1362990348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0cc620 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 -- 172.21.15.83:0/1362990348 >> 172.21.15.83:0/1362990348 conn(0x7f439c0b2b60 msgr2=0x7f439c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 -- 172.21.15.83:0/1362990348 shutdown_connections 2023-11-28T11:11:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.649+0000 7f43a2b41700 1 -- 172.21.15.83:0/1362990348 wait complete. 2023-11-28T11:11:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.650+0000 7f43a2b41700 1 Processor -- start 2023-11-28T11:11:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.650+0000 7f43a2b41700 1 -- start start 2023-11-28T11:11:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a2b41700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 0x7f439c09a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a2b41700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0ccb60 0x7f439c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a2b41700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a2b41700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c09b360 con 0x7f439c0aefd0 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a2b41700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c09b640 con 0x7f439c0cd1c0 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a2b41700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c09b920 con 0x7f439c0ccb60 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 0x7f439c09a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 0x7f439c09a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52088/0 (socket says 172.21.15.83:52088) 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 -- 172.21.15.83:0/2729953909 learned_addr learned my addr 172.21.15.83:0/2729953909 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a10de700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f439bfff700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0ccb60 0x7f439c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 -- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0ccb60 msgr2=0x7f439c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0ccb60 0x7f439c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 -- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 msgr2=0x7f439c0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.651+0000 7f43a08dd700 1 -- 172.21.15.83:0/2729953909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4390009500 con 0x7f439c0aefd0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.652+0000 7f43a08dd700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 0x7f439c09a880 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f439000ebb0 tx=0x7f439000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.652+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390005a20 con 0x7f439c0aefd0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.652+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390005b80 con 0x7f439c0aefd0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.652+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43880094a0 con 0x7f439c0aefd0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.652+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439000c850 con 0x7f439c0aefd0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.652+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c0b50e0 con 0x7f439c0aefd0 2023-11-28T11:11:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.653+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c0b4e00 con 0x7f439c0aefd0 2023-11-28T11:11:06.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.653+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4390022070 con 0x7f439c0aefd0 2023-11-28T11:11:06.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.653+0000 7f43997fa700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f43800682f0 0x7f438006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.654+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4390085f30 con 0x7f439c0aefd0 2023-11-28T11:11:06.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.654+0000 7f439bfff700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f43800682f0 0x7f438006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.654+0000 7f439bfff700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f43800682f0 0x7f438006a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f4388008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.658+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43900525d0 con 0x7f439c0aefd0 2023-11-28T11:11:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.837+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f439c09c250 con 0x7f439c0aefd0 2023-11-28T11:11:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.838+0000 7f43997fa700 1 -- 172.21.15.83:0/2729953909 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4390020020 con 0x7f439c0aefd0 2023-11-28T11:11:06.839 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:11:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f43800682f0 msgr2=0x7f438006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f43800682f0 0x7f438006a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f4388008030 comp rx=0 tx=0).stop 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 msgr2=0x7f439c09a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 0x7f439c09a880 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f439000ebb0 tx=0x7f439000eec0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 shutdown_connections 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f439c0cd1c0 0x7f439c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f43800682f0 0x7f438006a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439c0aefd0 0x7f439c09a880 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 --2- 172.21.15.83:0/2729953909 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f439c0ccb60 0x7f439c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 >> 172.21.15.83:0/2729953909 conn(0x7f439c0b2b60 msgr2=0x7f439c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 shutdown_connections 2023-11-28T11:11:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.840+0000 7f43a2b41700 1 -- 172.21.15.83:0/2729953909 wait complete. 2023-11-28T11:11:06.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:11:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.975+0000 7fc42100a700 1 Processor -- start 2023-11-28T11:11:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.975+0000 7fc42100a700 1 -- start start 2023-11-28T11:11:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc42100a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc41c15c890 0x7fc41c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc42100a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc41c0b4bb0 0x7fc41c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc42100a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41c06d4a0 con 0x7fc41c06e1a0 2023-11-28T11:11:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc42100a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c06e620 con 0x7fc41c0b4bb0 2023-11-28T11:11:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc42100a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c0b5020 con 0x7fc41c15c890 2023-11-28T11:11:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc41c0b4bb0 0x7fc41c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41b59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc41c06e1a0 0x7fc41c1604e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49958/0 (socket says 172.21.15.83:49958) 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41b59e700 1 -- 172.21.15.83:0/4229768395 learned_addr learned my addr 172.21.15.83:0/4229768395 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc41c0b4bb0 0x7fc41c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52094/0 (socket says 172.21.15.83:52094) 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41ad9d700 1 --2- 172.21.15.83:0/4229768395 >> v2:172.21.15.90:3300/0 conn(0x7fc41c15c890 0x7fc41c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 -- 172.21.15.83:0/4229768395 >> v2:172.21.15.90:3300/0 conn(0x7fc41c15c890 msgr2=0x7fc41c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 --2- 172.21.15.83:0/4229768395 >> v2:172.21.15.90:3300/0 conn(0x7fc41c15c890 0x7fc41c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 -- 172.21.15.83:0/4229768395 >> v1:172.21.15.83:6789/0 conn(0x7fc41c06e1a0 legacy=0x7fc41c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 -- 172.21.15.83:0/4229768395 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c06c160 con 0x7fc41c0b4bb0 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.976+0000 7fc41a59c700 1 --2- 172.21.15.83:0/4229768395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c0b4bb0 0x7fc41c06dc60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0715b0 tx=0x7fc40c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fc6ffc1647ac83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc419d9b700 1 -- 172.21.15.83:0/4229768395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c011830 con 0x7fc41c0b4bb0 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc419d9b700 1 -- 172.21.15.83:0/4229768395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c011990 con 0x7fc41c0b4bb0 2023-11-28T11:11:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc419d9b700 1 -- 172.21.15.83:0/4229768395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c011d20 con 0x7fc41c0b4bb0 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 -- 172.21.15.83:0/4229768395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c0b4bb0 msgr2=0x7fc41c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 --2- 172.21.15.83:0/4229768395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c0b4bb0 0x7fc41c06dc60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0715b0 tx=0x7fc40c009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 -- 172.21.15.83:0/4229768395 shutdown_connections 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 --2- 172.21.15.83:0/4229768395 >> v2:172.21.15.90:3300/0 conn(0x7fc41c15c890 0x7fc41c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 --2- 172.21.15.83:0/4229768395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c0b4bb0 0x7fc41c06dc60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 -- 172.21.15.83:0/4229768395 >> 172.21.15.83:0/4229768395 conn(0x7fc41c0b2b60 msgr2=0x7fc41c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 -- 172.21.15.83:0/4229768395 shutdown_connections 2023-11-28T11:11:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.977+0000 7fc42100a700 1 -- 172.21.15.83:0/4229768395 wait complete. 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 Processor -- start 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 -- start start 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 0x7fc41c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc41c0b4bb0 0x7fc41c153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc41c15c890 0x7fc41c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c1545b0 con 0x7fc41c06e1a0 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0ccc30 con 0x7fc41c15c890 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc42100a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0adb10 con 0x7fc41c0b4bb0 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc41ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 0x7fc41c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc41ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 0x7fc41c0cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52100/0 (socket says 172.21.15.83:52100) 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.979+0000 7fc41ad9d700 1 -- 172.21.15.83:0/1390554951 learned_addr learned my addr 172.21.15.83:0/1390554951 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41a59c700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc41c0b4bb0 0x7fc41c153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41ad9d700 1 -- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc41c0b4bb0 msgr2=0x7fc41c153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41b59e700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc41c15c890 0x7fc41c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41ad9d700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc41c0b4bb0 0x7fc41c153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41ad9d700 1 -- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc41c15c890 msgr2=0x7fc41c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41ad9d700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc41c15c890 0x7fc41c0c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41ad9d700 1 -- 172.21.15.83:0/1390554951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0094a0 con 0x7fc41c06e1a0 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc41ad9d700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 0x7fc41c0cc570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc41000e730 tx=0x7fc41000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410013610 con 0x7fc41c06e1a0 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410013770 con 0x7fc41c06e1a0 2023-11-28T11:11:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c06c400 con 0x7fc41c06e1a0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410013b20 con 0x7fc41c06e1a0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.980+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c0b5ae0 con 0x7fc41c06e1a0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.981+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c0b5240 con 0x7fc41c06e1a0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.981+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc410019030 con 0x7fc41c06e1a0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.982+0000 7fc4037fe700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4080682f0 0x7fc40806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.982+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc410085770 con 0x7fc41c06e1a0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.982+0000 7fc41a59c700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4080682f0 0x7fc40806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:06.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.982+0000 7fc41a59c700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4080682f0 0x7fc40806a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b1630 tx=0x7fc40c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:06.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:06.986+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc410051e90 con 0x7fc41c06e1a0 2023-11-28T11:11:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.164+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc41c0742b0 con 0x7fc41c06e1a0 2023-11-28T11:11:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.165+0000 7fc4037fe700 1 -- 172.21.15.83:0/1390554951 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4100554b0 con 0x7fc41c06e1a0 2023-11-28T11:11:07.166 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:11:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4080682f0 msgr2=0x7fc40806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4080682f0 0x7fc40806a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b1630 tx=0x7fc40c008030 comp rx=0 tx=0).stop 2023-11-28T11:11:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 msgr2=0x7fc41c0cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 0x7fc41c0cc570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc41000e730 tx=0x7fc41000eaf0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 shutdown_connections 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc41c15c890 0x7fc41c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc4080682f0 0x7fc40806a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c06e1a0 0x7fc41c0cc570 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 --2- 172.21.15.83:0/1390554951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc41c0b4bb0 0x7fc41c153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 >> 172.21.15.83:0/1390554951 conn(0x7fc41c0b2b60 msgr2=0x7fc41c0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 shutdown_connections 2023-11-28T11:11:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.167+0000 7fc42100a700 1 -- 172.21.15.83:0/1390554951 wait complete. 2023-11-28T11:11:07.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:11:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.301+0000 7f6484a08700 1 Processor -- start 2023-11-28T11:11:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.302+0000 7f6484a08700 1 -- start start 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.302+0000 7f6484a08700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f64800b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.302+0000 7f6484a08700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648015ec90 con 0x7f64800b1bd0 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.302+0000 7f6484a08700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64800cb270 con 0x7f648006bbf0 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.302+0000 7f6484a08700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648015e380 con 0x7f64800cbe30 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647dd9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f64800b1bd0 0x7f648006b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49970/0 (socket says 172.21.15.83:49970) 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647dd9b700 1 -- 172.21.15.83:0/662842936 learned_addr learned my addr 172.21.15.83:0/662842936 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647e59c700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f64800b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069069965 0 0) 0x7f648015ec90 con 0x7f64800b1bd0 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64680035b0 con 0x7f64800b1bd0 2023-11-28T11:11:07.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939438270 0 0) 0x7f64800cb270 con 0x7f648006bbf0 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f648015ec90 con 0x7f648006bbf0 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3027937727 0 0) 0x7f64680035b0 con 0x7f64800b1bd0 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 msgr2=0x7f64800b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f64800b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 >> v1:172.21.15.90:6789/0 conn(0x7f648006bbf0 legacy=0x7f648015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800b2110 con 0x7f64800b1bd0 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6470001670 con 0x7f64800b1bd0 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647e59c700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f64800b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 >> v1:172.21.15.83:6789/0 conn(0x7f64800b1bd0 legacy=0x7f648006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468003bc0 0x7f6468003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f6468004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6468004d90 0x7f6468005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648015e380 con 0x7f6468004d90 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646800bac0 con 0x7f6468003bc0 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646800bc00 con 0x7f64800cbe30 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647ed9d700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6468004d90 0x7f6468005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647e59c700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468003bc0 0x7f6468003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.303+0000 7f647dd9b700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f6468004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 -- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 msgr2=0x7f6468004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64800cbe30 0x7f6468004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 -- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468003bc0 msgr2=0x7f6468003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6468003bc0 0x7f6468003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 -- 172.21.15.83:0/662842936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64740094a0 con 0x7f6468004d90 2023-11-28T11:11:07.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6468004d90 0x7f6468005200 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6478002230 tx=0x7f6468005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79163c64e156e8a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478001f30 con 0x7f6468004d90 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647800ce60 con 0x7f6468004d90 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647d59a700 1 -- 172.21.15.83:0/662842936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478003c40 con 0x7f6468004d90 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f6484a08700 1 -- 172.21.15.83:0/662842936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6468004d90 msgr2=0x7f6468005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f6484a08700 1 --2- 172.21.15.83:0/662842936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6468004d90 0x7f6468005200 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6478002230 tx=0x7f6468005880 comp rx=0 tx=0).stop 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f647ed9d700 1 -- 172.21.15.83:0/662842936 reap_dead start 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f6484a08700 1 -- 172.21.15.83:0/662842936 shutdown_connections 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f6484a08700 1 -- 172.21.15.83:0/662842936 >> 172.21.15.83:0/662842936 conn(0x7f64800cc690 msgr2=0x7f64800cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f6484a08700 1 -- 172.21.15.83:0/662842936 shutdown_connections 2023-11-28T11:11:07.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.304+0000 7f6484a08700 1 -- 172.21.15.83:0/662842936 wait complete. 2023-11-28T11:11:07.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.306+0000 7f6484a08700 1 Processor -- start 2023-11-28T11:11:07.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.306+0000 7f6484a08700 1 -- start start 2023-11-28T11:11:07.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6468004d90 0x7f6480157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64800af040 0x7f6480157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 0x7f64800b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800b3440 con 0x7f64800b00e0 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800b0aa0 con 0x7f64800af040 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64800af4b0 con 0x7f6468004d90 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 0x7f64800b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 0x7f64800b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52124/0 (socket says 172.21.15.83:52124) 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 -- 172.21.15.83:0/3671705589 learned_addr learned my addr 172.21.15.83:0/3671705589 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647e59c700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6468004d90 0x7f6480157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647dd9b700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64800af040 0x7f6480157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 -- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6468004d90 msgr2=0x7f6480157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6468004d90 0x7f6480157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 -- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64800af040 msgr2=0x7f6480157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64800af040 0x7f6480157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 -- 172.21.15.83:0/3671705589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648015e920 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647dd9b700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64800af040 0x7f6480157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f647ed9d700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 0x7f64800b0560 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6478005040 tx=0x7f6478001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800cc60 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780050d0 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648015e640 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478005480 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.307+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64800cd3c0 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.308+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64800ceb00 con 0x7f64800b00e0 2023-11-28T11:11:07.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.309+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f647802e070 con 0x7f64800b00e0 2023-11-28T11:11:07.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.309+0000 7f646effd700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64600682f0 0x7f646006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.309+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6478086e60 con 0x7f64800b00e0 2023-11-28T11:11:07.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.309+0000 7f647e59c700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64600682f0 0x7f646006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.310+0000 7f647e59c700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64600682f0 0x7f646006a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f64800b5b90 tx=0x7f6474008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.313+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6478053580 con 0x7f64800b00e0 2023-11-28T11:11:07.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.492+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64800742b0 con 0x7f64800b00e0 2023-11-28T11:11:07.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.493+0000 7f646effd700 1 -- 172.21.15.83:0/3671705589 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6478017020 con 0x7f64800b00e0 2023-11-28T11:11:07.494 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:11:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64600682f0 msgr2=0x7f646006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64600682f0 0x7f646006a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f64800b5b90 tx=0x7f6474008040 comp rx=0 tx=0).stop 2023-11-28T11:11:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 msgr2=0x7f64800b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 0x7f64800b0560 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6478005040 tx=0x7f6478001650 comp rx=0 tx=0).stop 2023-11-28T11:11:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 shutdown_connections 2023-11-28T11:11:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f64600682f0 0x7f646006a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64800b00e0 0x7f64800b0560 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6468004d90 0x7f6480157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.494+0000 7f6484a08700 1 --2- 172.21.15.83:0/3671705589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64800af040 0x7f6480157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.495+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 >> 172.21.15.83:0/3671705589 conn(0x7f64800cc690 msgr2=0x7f6480158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.495+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 shutdown_connections 2023-11-28T11:11:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.495+0000 7f6484a08700 1 -- 172.21.15.83:0/3671705589 wait complete. 2023-11-28T11:11:07.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:11:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.630+0000 7f8872396700 1 Processor -- start 2023-11-28T11:11:07.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.630+0000 7f8872396700 1 -- start start 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8872396700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f886c0cc8a0 0x7f886c1639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8872396700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0c9e70 con 0x7f886c158cd0 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8872396700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c15c790 con 0x7f886c158900 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8872396700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886c15ca70 con 0x7f886c0cc8a0 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8870933700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f886c0cc8a0 0x7f886c1639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8870933700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f886c0cc8a0 0x7f886c1639f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52132/0 (socket says 172.21.15.83:52132) 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8870933700 1 -- 172.21.15.83:0/2568071928 learned_addr learned my addr 172.21.15.83:0/2568071928 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f886b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f886c158cd0 0x7f886c0cc190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49988/0 (socket says 172.21.15.83:49988) 2023-11-28T11:11:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8870933700 1 -- 172.21.15.83:0/2568071928 >> v1:172.21.15.90:6789/0 conn(0x7f886c158900 legacy=0x7f886c159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8870933700 1 -- 172.21.15.83:0/2568071928 >> v1:172.21.15.83:6789/0 conn(0x7f886c158cd0 legacy=0x7f886c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.631+0000 7f8870933700 1 -- 172.21.15.83:0/2568071928 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c15cd50 con 0x7f886c0cc8a0 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8870933700 1 --2- 172.21.15.83:0/2568071928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c0cc8a0 0x7f886c1639f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8864009e50 tx=0x7f886400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f500d707404acc1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f886affd700 1 -- 172.21.15.83:0/2568071928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400e070 con 0x7f886c0cc8a0 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f886affd700 1 -- 172.21.15.83:0/2568071928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864003a20 con 0x7f886c0cc8a0 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f886affd700 1 -- 172.21.15.83:0/2568071928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864003dd0 con 0x7f886c0cc8a0 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 -- 172.21.15.83:0/2568071928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c0cc8a0 msgr2=0x7f886c1639f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 --2- 172.21.15.83:0/2568071928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c0cc8a0 0x7f886c1639f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8864009e50 tx=0x7f886400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 -- 172.21.15.83:0/2568071928 shutdown_connections 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 --2- 172.21.15.83:0/2568071928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c0cc8a0 0x7f886c1639f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 -- 172.21.15.83:0/2568071928 >> 172.21.15.83:0/2568071928 conn(0x7f886c0b1090 msgr2=0x7f886c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 -- 172.21.15.83:0/2568071928 shutdown_connections 2023-11-28T11:11:07.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.632+0000 7f8872396700 1 -- 172.21.15.83:0/2568071928 wait complete. 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.633+0000 7f8872396700 1 Processor -- start 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 -- start start 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f886c0cc8a0 0x7f886c1f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f886c158900 0x7f886c154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 0x7f886c155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c1567e0 con 0x7f886c158cd0 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c1559b0 con 0x7f886c158900 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.634+0000 7f8872396700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c155c60 con 0x7f886c0cc8a0 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 0x7f886c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 0x7f886c155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52138/0 (socket says 172.21.15.83:52138) 2023-11-28T11:11:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 -- 172.21.15.83:0/1045341389 learned_addr learned my addr 172.21.15.83:0/1045341389 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f886b7fe700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f886c158900 0x7f886c154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 -- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f886c0cc8a0 msgr2=0x7f886c1f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f886c0cc8a0 0x7f886c1f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 -- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f886c158900 msgr2=0x7f886c154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f886c158900 0x7f886c154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 -- 172.21.15.83:0/1045341389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864009b30 con 0x7f886c158cd0 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f886b7fe700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f886c158900 0x7f886c154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8870933700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 0x7f886c155470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8864004000 tx=0x7f8864004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864004440 con 0x7f886c158cd0 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88640045a0 con 0x7f886c158cd0 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c15d310 con 0x7f886c158cd0 2023-11-28T11:11:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864004930 con 0x7f886c158cd0 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.635+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c15d030 con 0x7f886c158cd0 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.636+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c0ce040 con 0x7f886c158cd0 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.636+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f886401d440 con 0x7f886c158cd0 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.637+0000 7f8868ff9700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88580682f0 0x7f885806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.637+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8864087430 con 0x7f886c158cd0 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.637+0000 7f886bfff700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88580682f0 0x7f885806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.637+0000 7f886bfff700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88580682f0 0x7f885806a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f885c000f60 tx=0x7f885c007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.641+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8864053af0 con 0x7f886c158cd0 2023-11-28T11:11:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.824+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f886c0742b0 con 0x7f886c158cd0 2023-11-28T11:11:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.825+0000 7f8868ff9700 1 -- 172.21.15.83:0/1045341389 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8864057110 con 0x7f886c158cd0 2023-11-28T11:11:07.827 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:11:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88580682f0 msgr2=0x7f885806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88580682f0 0x7f885806a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f885c000f60 tx=0x7f885c007a40 comp rx=0 tx=0).stop 2023-11-28T11:11:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 msgr2=0x7f886c155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 0x7f886c155470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8864004000 tx=0x7f8864004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 shutdown_connections 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88580682f0 0x7f885806a7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f886c158cd0 0x7f886c155470 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f886c0cc8a0 0x7f886c1f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 --2- 172.21.15.83:0/1045341389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f886c158900 0x7f886c154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.827+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 >> 172.21.15.83:0/1045341389 conn(0x7f886c0b1090 msgr2=0x7f886c071200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.828+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 shutdown_connections 2023-11-28T11:11:07.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.828+0000 7f8872396700 1 -- 172.21.15.83:0/1045341389 wait complete. 2023-11-28T11:11:07.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:11:07.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.962+0000 7f71323c0700 1 Processor -- start 2023-11-28T11:11:07.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.962+0000 7f71323c0700 1 -- start start 2023-11-28T11:11:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f71323c0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f712c0aefd0 0x7f712c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f71323c0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c0b4bb0 con 0x7f712c0ccd30 2023-11-28T11:11:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f71323c0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c0b4e00 con 0x7f712c0cd1c0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f71323c0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c0d0af0 con 0x7f712c0aefd0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f712c0aefd0 0x7f712c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f713095d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f712c0ccd30 0x7f712c0cd990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50006/0 (socket says 172.21.15.83:50006) 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f713095d700 1 -- 172.21.15.83:0/3064010366 learned_addr learned my addr 172.21.15.83:0/3064010366 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712bfff700 1 -- 172.21.15.83:0/3064010366 >> v1:172.21.15.90:6789/0 conn(0x7f712c0cd1c0 legacy=0x7f712c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712bfff700 1 -- 172.21.15.83:0/3064010366 >> v1:172.21.15.83:6789/0 conn(0x7f712c0ccd30 legacy=0x7f712c0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712bfff700 1 -- 172.21.15.83:0/3064010366 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0b50e0 con 0x7f712c0aefd0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712bfff700 1 --2- 172.21.15.83:0/3064010366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c0d05b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f711c009820 tx=0x7f711c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f72ee51a72ea4ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712affd700 1 -- 172.21.15.83:0/3064010366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c005e30 con 0x7f712c0aefd0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712affd700 1 -- 172.21.15.83:0/3064010366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c006030 con 0x7f712c0aefd0 2023-11-28T11:11:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.963+0000 7f712affd700 1 -- 172.21.15.83:0/3064010366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0063e0 con 0x7f712c0aefd0 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 -- 172.21.15.83:0/3064010366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 msgr2=0x7f712c0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 --2- 172.21.15.83:0/3064010366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c0d05b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f711c009820 tx=0x7f711c009b30 comp rx=0 tx=0).stop 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 -- 172.21.15.83:0/3064010366 shutdown_connections 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 --2- 172.21.15.83:0/3064010366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c0d05b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 -- 172.21.15.83:0/3064010366 >> 172.21.15.83:0/3064010366 conn(0x7f712c0b2b60 msgr2=0x7f712c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 -- 172.21.15.83:0/3064010366 shutdown_connections 2023-11-28T11:11:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.964+0000 7f71323c0700 1 -- 172.21.15.83:0/3064010366 wait complete. 2023-11-28T11:11:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.965+0000 7f71323c0700 1 Processor -- start 2023-11-28T11:11:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.965+0000 7f71323c0700 1 -- start start 2023-11-28T11:11:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f71323c0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f71323c0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f712c0ccd30 0x7f712c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f71323c0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f712c0cd1c0 0x7f712c06f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f71323c0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c15ba50 con 0x7f712c0aefd0 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f71323c0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c15c870 con 0x7f712c0ccd30 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f71323c0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c150b90 con 0x7f712c0cd1c0 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52148/0 (socket says 172.21.15.83:52148) 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 -- 172.21.15.83:0/3266568367 learned_addr learned my addr 172.21.15.83:0/3266568367 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 -- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f712c0cd1c0 msgr2=0x7f712c06f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712b7fe700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f712c0ccd30 0x7f712c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f712c0cd1c0 0x7f712c06f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 -- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f712c0ccd30 msgr2=0x7f712c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f712c0ccd30 0x7f712c06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.966+0000 7f712bfff700 1 -- 172.21.15.83:0/3266568367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c009430 con 0x7f712c0aefd0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.967+0000 7f712bfff700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c15cd00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.967+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0065b0 con 0x7f712c0aefd0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.967+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c006730 con 0x7f712c0aefd0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.967+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0b56a0 con 0x7f712c0aefd0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.967+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0036a0 con 0x7f712c0aefd0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.967+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c0b53c0 con 0x7f712c0aefd0 2023-11-28T11:11:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.968+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c0b5980 con 0x7f712c0aefd0 2023-11-28T11:11:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.969+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f711c00f050 con 0x7f712c0aefd0 2023-11-28T11:11:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.969+0000 7f7128ff9700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71180682f0 0x7f711806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.969+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f711c087cb0 con 0x7f712c0aefd0 2023-11-28T11:11:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.969+0000 7f712b7fe700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71180682f0 0x7f711806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.970+0000 7f712b7fe700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71180682f0 0x7f711806a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f712c0b05d0 tx=0x7f7120009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:07.973+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f711c0543f0 con 0x7f712c0aefd0 2023-11-28T11:11:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.173+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f712c0742b0 con 0x7f712c0aefd0 2023-11-28T11:11:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.173+0000 7f7128ff9700 1 -- 172.21.15.83:0/3266568367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f711c057a10 con 0x7f712c0aefd0 2023-11-28T11:11:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71180682f0 msgr2=0x7f711806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71180682f0 0x7f711806a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f712c0b05d0 tx=0x7f7120009040 comp rx=0 tx=0).stop 2023-11-28T11:11:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 msgr2=0x7f712c15cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c15cd00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c004750 comp rx=0 tx=0).stop 2023-11-28T11:11:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 shutdown_connections 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f712c0cd1c0 0x7f712c06f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71180682f0 0x7f711806a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f712c0aefd0 0x7f712c15cd00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 --2- 172.21.15.83:0/3266568367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f712c0ccd30 0x7f712c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 >> 172.21.15.83:0/3266568367 conn(0x7f712c0b2b60 msgr2=0x7f712c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 shutdown_connections 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.175+0000 7f71323c0700 1 -- 172.21.15.83:0/3266568367 wait complete. 2023-11-28T11:11:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:11:08.189 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:11:08.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:08.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:08.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:11:08.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:08.261 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:11:08.284 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:11:08.284 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:08.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:08.304 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:11:08.304 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:08.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:11:08.342 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:11:08.342 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:08.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:08.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.500+0000 7f88ddedd700 1 Processor -- start 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.500+0000 7f88ddedd700 1 -- start start 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88ddedd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88d80b1fb0 0x7f88d80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88ddedd700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d80cbfb0 con 0x7f88d815d0e0 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88ddedd700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d80cb710 con 0x7f88d815cd10 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88ddedd700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d80cb9f0 con 0x7f88d80b1fb0 2023-11-28T11:11:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88d80b1fb0 0x7f88d80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d6ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f88d815d0e0 0x7f88d806bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50018/0 (socket says 172.21.15.83:50018) 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d6ffd700 1 -- 172.21.15.83:0/410940938 learned_addr learned my addr 172.21.15.83:0/410940938 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d7fff700 1 -- 172.21.15.83:0/410940938 >> v1:172.21.15.90:6789/0 conn(0x7f88d815cd10 legacy=0x7f88d80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d7fff700 1 -- 172.21.15.83:0/410940938 >> v1:172.21.15.83:6789/0 conn(0x7f88d815d0e0 legacy=0x7f88d806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d7fff700 1 -- 172.21.15.83:0/410940938 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80cbcd0 con 0x7f88d80b1fb0 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d7fff700 1 --2- 172.21.15.83:0/410940938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d80b1fb0 0x7f88d80ca970 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009d90 tx=0x7f88d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8266877ba8f9ad80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d67fc700 1 -- 172.21.15.83:0/410940938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d000e070 con 0x7f88d80b1fb0 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d67fc700 1 -- 172.21.15.83:0/410940938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d000bb90 con 0x7f88d80b1fb0 2023-11-28T11:11:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.501+0000 7f88d67fc700 1 -- 172.21.15.83:0/410940938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0003a20 con 0x7f88d80b1fb0 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 -- 172.21.15.83:0/410940938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d80b1fb0 msgr2=0x7f88d80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 --2- 172.21.15.83:0/410940938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d80b1fb0 0x7f88d80ca970 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009d90 tx=0x7f88d000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 -- 172.21.15.83:0/410940938 shutdown_connections 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 --2- 172.21.15.83:0/410940938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d80b1fb0 0x7f88d80ca970 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 -- 172.21.15.83:0/410940938 >> 172.21.15.83:0/410940938 conn(0x7f88d806e9a0 msgr2=0x7f88d80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 -- 172.21.15.83:0/410940938 shutdown_connections 2023-11-28T11:11:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.502+0000 7f88ddedd700 1 -- 172.21.15.83:0/410940938 wait complete. 2023-11-28T11:11:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.503+0000 7f88ddedd700 1 Processor -- start 2023-11-28T11:11:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.503+0000 7f88ddedd700 1 -- start start 2023-11-28T11:11:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88ddedd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88d80b1fb0 0x7f88d80c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88ddedd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 0x7f88d81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88ddedd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88d815d0e0 0x7f88d80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88ddedd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80c5ec0 con 0x7f88d815cd10 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88ddedd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80c67d0 con 0x7f88d80b1fb0 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88ddedd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80c4ec0 con 0x7f88d815d0e0 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 0x7f88d81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 0x7f88d81563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52164/0 (socket says 172.21.15.83:52164) 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 -- 172.21.15.83:0/1227033331 learned_addr learned my addr 172.21.15.83:0/1227033331 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d77fe700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88d80b1fb0 0x7f88d80c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d7fff700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88d815d0e0 0x7f88d80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 -- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88d815d0e0 msgr2=0x7f88d80cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88d815d0e0 0x7f88d80cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 -- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88d80b1fb0 msgr2=0x7f88d80c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88d80b1fb0 0x7f88d80c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d6ffd700 1 -- 172.21.15.83:0/1227033331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009a70 con 0x7f88d815cd10 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.504+0000 7f88d77fe700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88d80b1fb0 0x7f88d80c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.505+0000 7f88d6ffd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 0x7f88d81563b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c460 tx=0x7f88c800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.505+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8005d70 con 0x7f88d815cd10 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.505+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c80070d0 con 0x7f88d815cd10 2023-11-28T11:11:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.505+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d806b660 con 0x7f88d815cd10 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.505+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8007460 con 0x7f88d815cd10 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.505+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d806b9b0 con 0x7f88d815cd10 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.506+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d80cd9f0 con 0x7f88d815cd10 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.506+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88c8019030 con 0x7f88d815cd10 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.506+0000 7f88cffff700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88b80682f0 0x7f88b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.507+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f88c8085160 con 0x7f88d815cd10 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.507+0000 7f88d77fe700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88b80682f0 0x7f88b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.507+0000 7f88d77fe700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88b80682f0 0x7f88b806a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f88c0003fc0 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.511+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88c8051800 con 0x7f88d815cd10 2023-11-28T11:11:08.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.735+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88d8074320 con 0x7f88d815cd10 2023-11-28T11:11:08.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.736+0000 7f88cffff700 1 -- 172.21.15.83:0/1227033331 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f88c8054e20 con 0x7f88d815cd10 2023-11-28T11:11:08.737 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:08.738 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:08.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88b80682f0 msgr2=0x7f88b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88b80682f0 0x7f88b806a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f88c0003fc0 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2023-11-28T11:11:08.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 msgr2=0x7f88d81563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 0x7f88d81563b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c460 tx=0x7f88c800c820 comp rx=0 tx=0).stop 2023-11-28T11:11:08.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 shutdown_connections 2023-11-28T11:11:08.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88d815d0e0 0x7f88d80cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88d80b1fb0 0x7f88d80c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f88b80682f0 0x7f88b806a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 --2- 172.21.15.83:0/1227033331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d815cd10 0x7f88d81563b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 >> 172.21.15.83:0/1227033331 conn(0x7f88d806e9a0 msgr2=0x7f88d80caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:08.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 shutdown_connections 2023-11-28T11:11:08.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.738+0000 7f88ddedd700 1 -- 172.21.15.83:0/1227033331 wait complete. 2023-11-28T11:11:08.742 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:11:08.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.870+0000 7fa6ba78e700 1 Processor -- start 2023-11-28T11:11:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.870+0000 7fa6ba78e700 1 -- start start 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6ba78e700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa6b40b1fb0 0x7fa6b40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6ba78e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40cbfb0 con 0x7fa6b415d0e0 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6ba78e700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40cb710 con 0x7fa6b415cd10 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6ba78e700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b40cb9f0 con 0x7fa6b40b1fb0 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6b37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa6b415d0e0 0x7fa6b406bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50032/0 (socket says 172.21.15.83:50032) 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6b37fe700 1 -- 172.21.15.83:0/1632170824 learned_addr learned my addr 172.21.15.83:0/1632170824 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.871+0000 7fa6b8d2b700 1 --2- 172.21.15.83:0/1632170824 >> v2:172.21.15.90:3300/0 conn(0x7fa6b40b1fb0 0x7fa6b40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989472539 0 0) 0x7fa6b40cbfb0 con 0x7fa6b415d0e0 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a00035b0 con 0x7fa6b415d0e0 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633880370 0 0) 0x7fa6b40cb710 con 0x7fa6b415cd10 2023-11-28T11:11:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b40cbfb0 con 0x7fa6b415cd10 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976373413 0 0) 0x7fa6a00035b0 con 0x7fa6b415d0e0 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 >> v2:172.21.15.90:3300/0 conn(0x7fa6b40b1fb0 msgr2=0x7fa6b40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 --2- 172.21.15.83:0/1632170824 >> v2:172.21.15.90:3300/0 conn(0x7fa6b40b1fb0 0x7fa6b40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 >> v1:172.21.15.90:6789/0 conn(0x7fa6b415cd10 legacy=0x7fa6b40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40cbcd0 con 0x7fa6b415d0e0 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fa69c001670 con 0x7fa6b415d0e0 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 >> v1:172.21.15.83:6789/0 conn(0x7fa6b415d0e0 legacy=0x7fa6b406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0003b90 0x7fa6a0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6a0004850 0x7fa6a0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6a0005200 0x7fa6a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40cb9f0 con 0x7fa6a0004850 2023-11-28T11:11:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a000c060 con 0x7fa6a0005200 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a000c220 con 0x7fa6a0003b90 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6a0004850 0x7fa6a0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b8d2b700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6a0005200 0x7fa6a000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b3fff700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0003b90 0x7fa6a0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 -- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0003b90 msgr2=0x7fa6a0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0003b90 0x7fa6a0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 -- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6a0005200 msgr2=0x7fa6a000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6a0005200 0x7fa6a000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b8d2b700 1 -- 172.21.15.83:0/1632170824 reap_dead start 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 -- 172.21.15.83:0/1632170824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00035b0 con 0x7fa6a0004850 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.872+0000 7fa6b37fe700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6a0004850 0x7fa6a0004cc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c210 tx=0x7fa6a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f5c229a967bcbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00cd10 con 0x7fa6a0004850 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c00ce90 con 0x7fa6a0004850 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6b2ffd700 1 -- 172.21.15.83:0/1632170824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c005ba0 con 0x7fa6a0004850 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1632170824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6a0004850 msgr2=0x7fa6a0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6a0004850 0x7fa6a0004cc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c210 tx=0x7fa6a0005880 comp rx=0 tx=0).stop 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1632170824 shutdown_connections 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1632170824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6a0004850 0x7fa6a0004cc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1632170824 >> 172.21.15.83:0/1632170824 conn(0x7fa6b406e9a0 msgr2=0x7fa6b40cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1632170824 shutdown_connections 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.873+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1632170824 wait complete. 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.874+0000 7fa6ba78e700 1 Processor -- start 2023-11-28T11:11:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 -- start start 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0004850 0x7fa6b41f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b415d4b0 0x7fa6b40cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 0x7fa6b415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40c9940 con 0x7fa6b415ea10 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b41f3810 con 0x7fa6b415d4b0 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6ba78e700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b415d970 con 0x7fa6a0004850 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6b8d2b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 0x7fa6b415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6b8d2b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 0x7fa6b415ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52186/0 (socket says 172.21.15.83:52186) 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6b8d2b700 1 -- 172.21.15.83:0/1696081845 learned_addr learned my addr 172.21.15.83:0/1696081845 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6b3fff700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0004850 0x7fa6b41f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.875+0000 7fa6b37fe700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b415d4b0 0x7fa6b40cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.876+0000 7fa6b8d2b700 1 -- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0004850 msgr2=0x7fa6b41f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.876+0000 7fa6b8d2b700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0004850 0x7fa6b41f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.876+0000 7fa6b8d2b700 1 -- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b415d4b0 msgr2=0x7fa6b40cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.876+0000 7fa6b8d2b700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b415d4b0 0x7fa6b40cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.876+0000 7fa6b8d2b700 1 -- 172.21.15.83:0/1696081845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b406b660 con 0x7fa6b415ea10 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.876+0000 7fa6b8d2b700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 0x7fa6b415ee90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40c38f0 tx=0x7fa6ac011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.880+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00e040 con 0x7fa6b415ea10 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.880+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac00f040 con 0x7fa6b415ea10 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.880+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b406b9b0 con 0x7fa6b415ea10 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.880+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00fc40 con 0x7fa6b415ea10 2023-11-28T11:11:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.880+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b40cd9f0 con 0x7fa6b415ea10 2023-11-28T11:11:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.881+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b4074320 con 0x7fa6b415ea10 2023-11-28T11:11:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.881+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6ac00fda0 con 0x7fa6b415ea10 2023-11-28T11:11:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.881+0000 7fa6b0ff9700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa694068320 0x7fa69406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.882+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa6ac086450 con 0x7fa6b415ea10 2023-11-28T11:11:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.882+0000 7fa6b3fff700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa694068320 0x7fa69406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.882+0000 7fa6b3fff700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa694068320 0x7fa69406a7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4000c00 tx=0x7fa6a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:08.886+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6ac052ae0 con 0x7fa6b415ea10 2023-11-28T11:11:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.068+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6b40b5d10 con 0x7fa6b415ea10 2023-11-28T11:11:09.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.069+0000 7fa6b0ff9700 1 -- 172.21.15.83:0/1696081845 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18074 (secure 0 0 0) 0x7fa6ac056100 con 0x7fa6b415ea10 2023-11-28T11:11:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:09.071 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:27.384310+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:09.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa694068320 msgr2=0x7fa69406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa694068320 0x7fa69406a7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4000c00 tx=0x7fa6a4009040 comp rx=0 tx=0).stop 2023-11-28T11:11:09.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 msgr2=0x7fa6b415ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 0x7fa6b415ee90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40c38f0 tx=0x7fa6ac011ed0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 shutdown_connections 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa694068320 0x7fa69406a7d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6b415ea10 0x7fa6b415ee90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa6a0004850 0x7fa6b41f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 --2- 172.21.15.83:0/1696081845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b415d4b0 0x7fa6b40cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 >> 172.21.15.83:0/1696081845 conn(0x7fa6b406e9a0 msgr2=0x7fa6b40b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 shutdown_connections 2023-11-28T11:11:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.072+0000 7fa6ba78e700 1 -- 172.21.15.83:0/1696081845 wait complete. 2023-11-28T11:11:09.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:24.134118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:25.340097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:11:09.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:11:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.208+0000 7f03f996a700 1 Processor -- start 2023-11-28T11:11:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f996a700 1 -- start start 2023-11-28T11:11:09.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f996a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03f415b170 0x7f03f406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f996a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f406d4d0 con 0x7f03f40711e0 2023-11-28T11:11:09.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f996a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f406cea0 con 0x7f03f415a690 2023-11-28T11:11:09.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f996a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f415b610 con 0x7f03f415b170 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03f415b170 0x7f03f406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f27fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03f40711e0 0x7f03f415aa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50050/0 (socket says 172.21.15.83:50050) 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.209+0000 7f03f27fc700 1 -- 172.21.15.83:0/1721325294 learned_addr learned my addr 172.21.15.83:0/1721325294 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f37fe700 1 -- 172.21.15.83:0/1721325294 >> v1:172.21.15.90:6789/0 conn(0x7f03f415a690 legacy=0x7f03f4070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f37fe700 1 -- 172.21.15.83:0/1721325294 >> v1:172.21.15.83:6789/0 conn(0x7f03f40711e0 legacy=0x7f03f415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f37fe700 1 -- 172.21.15.83:0/1721325294 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f415b800 con 0x7f03f415b170 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f1ffb700 1 -- 172.21.15.83:0/1721325294 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333987244 0 0) 0x7f03f406d4d0 con 0x7f03f40711e0 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f37fe700 1 --2- 172.21.15.83:0/1721325294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f406d960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009d90 tx=0x7f03ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a85a2292c9c89c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f1ffb700 1 -- 172.21.15.83:0/1721325294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00e070 con 0x7f03f415b170 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f1ffb700 1 -- 172.21.15.83:0/1721325294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec00bbd0 con 0x7f03f415b170 2023-11-28T11:11:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.210+0000 7f03f1ffb700 1 -- 172.21.15.83:0/1721325294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec012430 con 0x7f03f415b170 2023-11-28T11:11:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 -- 172.21.15.83:0/1721325294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 msgr2=0x7f03f406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 --2- 172.21.15.83:0/1721325294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f406d960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009d90 tx=0x7f03ec00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 -- 172.21.15.83:0/1721325294 shutdown_connections 2023-11-28T11:11:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 --2- 172.21.15.83:0/1721325294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f406d960 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 -- 172.21.15.83:0/1721325294 >> 172.21.15.83:0/1721325294 conn(0x7f03f40ceb80 msgr2=0x7f03f40cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 -- 172.21.15.83:0/1721325294 shutdown_connections 2023-11-28T11:11:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.211+0000 7f03f996a700 1 -- 172.21.15.83:0/1721325294 wait complete. 2023-11-28T11:11:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.212+0000 7f03f996a700 1 Processor -- start 2023-11-28T11:11:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.212+0000 7f03f996a700 1 -- start start 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03f40711e0 0x7f03f414bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03f415a690 0x7f03f4152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f41530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41f31d0 con 0x7f03f415b170 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41f2550 con 0x7f03f415a690 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41f2830 con 0x7f03f40711e0 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f41530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f41530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52198/0 (socket says 172.21.15.83:52198) 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 -- 172.21.15.83:0/1759384128 learned_addr learned my addr 172.21.15.83:0/1759384128 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f27fc700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03f415a690 0x7f03f4152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f2ffd700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03f40711e0 0x7f03f414bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 -- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03f40711e0 msgr2=0x7f03f414bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03f40711e0 0x7f03f414bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 -- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03f415a690 msgr2=0x7f03f4152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03f415a690 0x7f03f4152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 -- 172.21.15.83:0/1759384128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec009a70 con 0x7f03f415b170 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f27fc700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03f415a690 0x7f03f4152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f37fe700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f41530b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f03ec014040 tx=0x7f03ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec01b810 con 0x7f03f415b170 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec020070 con 0x7f03f415b170 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f406a270 con 0x7f03f415b170 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.213+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec01bbb0 con 0x7f03f415b170 2023-11-28T11:11:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.214+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f406d160 con 0x7f03f415b170 2023-11-28T11:11:09.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.215+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f03ec01bd10 con 0x7f03f415b170 2023-11-28T11:11:09.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.215+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f406a400 con 0x7f03f415b170 2023-11-28T11:11:09.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.215+0000 7f03dbfff700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03e00682f0 0x7f03e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.215+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f03ec022070 con 0x7f03f415b170 2023-11-28T11:11:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.215+0000 7f03f2ffd700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03e00682f0 0x7f03e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.216+0000 7f03f2ffd700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03e00682f0 0x7f03e006a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f03f40b4ac0 tx=0x7f03e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.220+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03ec054c90 con 0x7f03f415b170 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.397+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03f4150120 con 0x7f03f415b170 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.397+0000 7f03dbfff700 1 -- 172.21.15.83:0/1759384128 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f03f4150120 con 0x7f03f415b170 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03e00682f0 msgr2=0x7f03e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03e00682f0 0x7f03e006a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f03f40b4ac0 tx=0x7f03e8009040 comp rx=0 tx=0).stop 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 msgr2=0x7f03f41530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f41530b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f03ec014040 tx=0x7f03ec004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 shutdown_connections 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f03e00682f0 0x7f03e006a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f415b170 0x7f03f41530b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03f40711e0 0x7f03f414bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 --2- 172.21.15.83:0/1759384128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03f415a690 0x7f03f4152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.399+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 >> 172.21.15.83:0/1759384128 conn(0x7f03f40ceb80 msgr2=0x7f03f415c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.400+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 shutdown_connections 2023-11-28T11:11:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.400+0000 7f03f996a700 1 -- 172.21.15.83:0/1759384128 wait complete. 2023-11-28T11:11:09.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:11:09.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.533+0000 7ff891457700 1 Processor -- start 2023-11-28T11:11:09.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff891457700 1 -- start start 2023-11-28T11:11:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff891457700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff88c0b87e0 0x7ff88c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff891457700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c160bc0 0x7ff88c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff891457700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c0b26e0 con 0x7ff88c0b4e20 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff891457700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c0b52a0 con 0x7ff88c0b87e0 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff891457700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0d0a30 con 0x7ff88c160bc0 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff88affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff88c0b87e0 0x7ff88c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff88affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff88c0b87e0 0x7ff88c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52202/0 (socket says 172.21.15.83:52202) 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.534+0000 7ff88affd700 1 -- 172.21.15.83:0/71129378 learned_addr learned my addr 172.21.15.83:0/71129378 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff88affd700 1 -- 172.21.15.83:0/71129378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c160bc0 msgr2=0x7ff88c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff88a7fc700 1 --2- 172.21.15.83:0/71129378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c160bc0 0x7ff88c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff88affd700 1 --2- 172.21.15.83:0/71129378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c160bc0 0x7ff88c0b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff88affd700 1 -- 172.21.15.83:0/71129378 >> v1:172.21.15.90:6789/0 conn(0x7ff88c0b4e20 legacy=0x7ff88c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff88affd700 1 -- 172.21.15.83:0/71129378 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0d0d10 con 0x7ff88c0b87e0 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff88affd700 1 --2- 172.21.15.83:0/71129378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c0b87e0 0x7ff88c0cf0b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff8800098f0 tx=0x7ff880009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2cad434b960cc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff889ffb700 1 -- 172.21.15.83:0/71129378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000b8a0 con 0x7ff88c0b87e0 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff889ffb700 1 -- 172.21.15.83:0/71129378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88000ba00 con 0x7ff88c0b87e0 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.535+0000 7ff889ffb700 1 -- 172.21.15.83:0/71129378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000bd90 con 0x7ff88c0b87e0 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 -- 172.21.15.83:0/71129378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c0b87e0 msgr2=0x7ff88c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 --2- 172.21.15.83:0/71129378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c0b87e0 0x7ff88c0cf0b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff8800098f0 tx=0x7ff880009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 -- 172.21.15.83:0/71129378 shutdown_connections 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 --2- 172.21.15.83:0/71129378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c0b87e0 0x7ff88c0cf0b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 --2- 172.21.15.83:0/71129378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c160bc0 0x7ff88c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 -- 172.21.15.83:0/71129378 >> 172.21.15.83:0/71129378 conn(0x7ff88c06cef0 msgr2=0x7ff88c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 -- 172.21.15.83:0/71129378 shutdown_connections 2023-11-28T11:11:09.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.536+0000 7ff891457700 1 -- 172.21.15.83:0/71129378 wait complete. 2023-11-28T11:11:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.537+0000 7ff891457700 1 Processor -- start 2023-11-28T11:11:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.537+0000 7ff891457700 1 -- start start 2023-11-28T11:11:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff891457700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c0b87e0 0x7ff88c1e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff891457700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff88c160bc0 0x7ff88c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff891457700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 0x7ff88c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff891457700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1e7ff0 con 0x7ff88c15c9d0 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff891457700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1e89c0 con 0x7ff88c160bc0 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff891457700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1e73e0 con 0x7ff88c0b87e0 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 0x7ff88c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 0x7ff88c15ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52212/0 (socket says 172.21.15.83:52212) 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 -- 172.21.15.83:0/871830651 learned_addr learned my addr 172.21.15.83:0/871830651 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88affd700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c0b87e0 0x7ff88c1e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88a7fc700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff88c160bc0 0x7ff88c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 -- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c0b87e0 msgr2=0x7ff88c1e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c0b87e0 0x7ff88c1e8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 -- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff88c160bc0 msgr2=0x7ff88c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff88c160bc0 0x7ff88c0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 -- 172.21.15.83:0/871830651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff880009550 con 0x7ff88c15c9d0 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff88b7fe700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 0x7ff88c15ce40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff88400cb50 tx=0x7ff88400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.538+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884003c00 con 0x7ff88c15c9d0 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.539+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff884003d60 con 0x7ff88c15c9d0 2023-11-28T11:11:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.539+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c161010 con 0x7ff88c15c9d0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.539+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88400aa20 con 0x7ff88c15c9d0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.539+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c0cd4b0 con 0x7ff88c15c9d0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.540+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c0cd750 con 0x7ff88c15c9d0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.540+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff884010050 con 0x7ff88c15c9d0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.540+0000 7ff8737fe700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff8780682f0 0x7ff87806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.540+0000 7ff88affd700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff8780682f0 0x7ff87806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.540+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff884085c40 con 0x7ff88c15c9d0 2023-11-28T11:11:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.541+0000 7ff88affd700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff8780682f0 0x7ff87806a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff880009840 tx=0x7ff880008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.545+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff884052360 con 0x7ff88c15c9d0 2023-11-28T11:11:09.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.727+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff88c0742b0 con 0x7ff88c15c9d0 2023-11-28T11:11:09.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.727+0000 7ff8737fe700 1 -- 172.21.15.83:0/871830651 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff884055980 con 0x7ff88c15c9d0 2023-11-28T11:11:09.729 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff8780682f0 msgr2=0x7ff87806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff8780682f0 0x7ff87806a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff880009840 tx=0x7ff880008030 comp rx=0 tx=0).stop 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 msgr2=0x7ff88c15ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 0x7ff88c15ce40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff88400cb50 tx=0x7ff88400ce60 comp rx=0 tx=0).stop 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 shutdown_connections 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff8780682f0 0x7ff87806a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff88c15c9d0 0x7ff88c15ce40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff88c0b87e0 0x7ff88c1e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 --2- 172.21.15.83:0/871830651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff88c160bc0 0x7ff88c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 >> 172.21.15.83:0/871830651 conn(0x7ff88c0b4e20 msgr2=0x7ff88c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 shutdown_connections 2023-11-28T11:11:09.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.729+0000 7ff891457700 1 -- 172.21.15.83:0/871830651 wait complete. 2023-11-28T11:11:09.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:11:09.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:09.787 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:09.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.866+0000 7f50abe90700 1 Processor -- start 2023-11-28T11:11:09.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.867+0000 7f50abe90700 1 -- start start 2023-11-28T11:11:09.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.867+0000 7f50abe90700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f50a406c370 0x7f50a40b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.867+0000 7f50abe90700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a40b5b00 con 0x7f50a40aee20 2023-11-28T11:11:09.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.867+0000 7f50abe90700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a406c800 con 0x7f50a40af1f0 2023-11-28T11:11:09.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.867+0000 7f50abe90700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a406cae0 con 0x7f50a406c370 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a9c2c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f50a40aee20 0x7f50a406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50076/0 (socket says 172.21.15.83:50076) 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a9c2c700 1 -- 172.21.15.83:0/1820792378 learned_addr learned my addr 172.21.15.83:0/1820792378 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a942b700 1 --2- 172.21.15.83:0/1820792378 >> v2:172.21.15.90:3300/0 conn(0x7f50a406c370 0x7f50a40b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911338736 0 0) 0x7f509c001610 con 0x7f50a40aee20 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50900035b0 con 0x7f50a40aee20 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260054745 0 0) 0x7f50900035b0 con 0x7f50a40aee20 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 >> v2:172.21.15.90:3300/0 conn(0x7f50a406c370 msgr2=0x7f50a40b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 --2- 172.21.15.83:0/1820792378 >> v2:172.21.15.90:3300/0 conn(0x7f50a406c370 0x7f50a40b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 >> v1:172.21.15.90:6789/0 conn(0x7f50a40af1f0 legacy=0x7f50a40cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a406cdc0 con 0x7f50a40aee20 2023-11-28T11:11:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f509c001d00 con 0x7f50a40aee20 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 >> v1:172.21.15.83:6789/0 conn(0x7f50a40aee20 legacy=0x7f50a406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5090003a00 0x7f5090003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090004850 0x7f5090004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5090005200 0x7f509000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a406cae0 con 0x7f5090003a00 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509000c110 con 0x7f5090004850 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509000c2d0 con 0x7f5090005200 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5090003a00 0x7f5090003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50a942b700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090004850 0x7f5090004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.868+0000 7f50aa42d700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5090005200 0x7f509000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a9c2c700 1 -- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5090005200 msgr2=0x7f509000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5090005200 0x7f509000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a9c2c700 1 -- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090004850 msgr2=0x7f5090004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090004850 0x7f5090004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a9c2c700 1 -- 172.21.15.83:0/1820792378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900035b0 con 0x7f5090003a00 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50aa42d700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5090005200 0x7f509000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50aa42d700 1 -- 172.21.15.83:0/1820792378 reap_dead start 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5090003a00 0x7f5090003e10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f509c002230 tx=0x7f5090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a0f665f2a13ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c005db0 con 0x7f5090003a00 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c00ae90 con 0x7f5090003a00 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50a8c2a700 1 -- 172.21.15.83:0/1820792378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c007310 con 0x7f5090003a00 2023-11-28T11:11:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50abe90700 1 -- 172.21.15.83:0/1820792378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5090003a00 msgr2=0x7f5090003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50abe90700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5090003a00 0x7f5090003e10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f509c002230 tx=0x7f5090005880 comp rx=0 tx=0).stop 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50abe90700 1 -- 172.21.15.83:0/1820792378 shutdown_connections 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50abe90700 1 --2- 172.21.15.83:0/1820792378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5090003a00 0x7f5090003e10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50abe90700 1 -- 172.21.15.83:0/1820792378 >> 172.21.15.83:0/1820792378 conn(0x7f50a40b2b60 msgr2=0x7f50a40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.869+0000 7f50abe90700 1 -- 172.21.15.83:0/1820792378 shutdown_connections 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.870+0000 7f50abe90700 1 -- 172.21.15.83:0/1820792378 wait complete. 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.871+0000 7f50abe90700 1 Processor -- start 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.871+0000 7f50abe90700 1 -- start start 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50abe90700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090003a00 0x7f50a4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50abe90700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50a40ce140 0x7f50a414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50abe90700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 0x7f50a414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50abe90700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41549e0 con 0x7f50a40ce5b0 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50abe90700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4155560 con 0x7f5090003a00 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50abe90700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4154c30 con 0x7f50a40ce140 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 0x7f50a414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 0x7f50a414e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52230/0 (socket says 172.21.15.83:52230) 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 -- 172.21.15.83:0/2495665414 learned_addr learned my addr 172.21.15.83:0/2495665414 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090003a00 0x7f50a4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50a942b700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50a40ce140 0x7f50a414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 -- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50a40ce140 msgr2=0x7f50a414e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50a40ce140 0x7f50a414e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 -- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090003a00 msgr2=0x7f50a4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.872+0000 7f50aa42d700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090003a00 0x7f50a4159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f50aa42d700 1 -- 172.21.15.83:0/2495665414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a406fc70 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f50aa42d700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 0x7f50a414e780 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f50a40cf160 tx=0x7f50a0010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0018070 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a00050d0 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a406d0a0 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0005460 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.873+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a406d380 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.874+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a41556f0 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.874+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f50a0003680 con 0x7f50a40ce5b0 2023-11-28T11:11:09.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.874+0000 7f509a7fc700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50880682f0 0x7f508806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.875+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f50a0087140 con 0x7f50a40ce5b0 2023-11-28T11:11:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.875+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50880682f0 0x7f508806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.875+0000 7f50a9c2c700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50880682f0 0x7f508806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f509c000f80 tx=0x7f509c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:09.879+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f50a0053860 con 0x7f50a40ce5b0 2023-11-28T11:11:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.062+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50a406bf40 con 0x7f50a40ce5b0 2023-11-28T11:11:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.062+0000 7f509a7fc700 1 -- 172.21.15.83:0/2495665414 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f50a0056e80 con 0x7f50a40ce5b0 2023-11-28T11:11:10.064 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:11:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.064+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50880682f0 msgr2=0x7f508806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.064+0000 7f50abe90700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50880682f0 0x7f508806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f509c000f80 tx=0x7f509c009000 comp rx=0 tx=0).stop 2023-11-28T11:11:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.064+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 msgr2=0x7f50a414e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.064+0000 7f50abe90700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 0x7f50a414e780 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f50a40cf160 tx=0x7f50a0010880 comp rx=0 tx=0).stop 2023-11-28T11:11:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 shutdown_connections 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50880682f0 0x7f508806a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50a40ce5b0 0x7f50a414e780 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5090003a00 0x7f50a4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 --2- 172.21.15.83:0/2495665414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50a40ce140 0x7f50a414e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 >> 172.21.15.83:0/2495665414 conn(0x7f50a40b2b60 msgr2=0x7f50a40cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 shutdown_connections 2023-11-28T11:11:10.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.065+0000 7f50abe90700 1 -- 172.21.15.83:0/2495665414 wait complete. 2023-11-28T11:11:10.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:11:10.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.199+0000 7fd170c6f700 1 Processor -- start 2023-11-28T11:11:10.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.199+0000 7fd170c6f700 1 -- start start 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd170c6f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd16c15c890 0x7fd16c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd170c6f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c06d4a0 con 0x7fd16c0b4bb0 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd170c6f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c0b5ae0 con 0x7fd16c06e370 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd170c6f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c0b5020 con 0x7fd16c15c890 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd16a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd16c15c890 0x7fd16c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd169d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd16c0b4bb0 0x7fd16c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50096/0 (socket says 172.21.15.83:50096) 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd169d9b700 1 -- 172.21.15.83:0/85417187 learned_addr learned my addr 172.21.15.83:0/85417187 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd16a59c700 1 -- 172.21.15.83:0/85417187 >> v1:172.21.15.90:6789/0 conn(0x7fd16c06e370 legacy=0x7fd16c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd16a59c700 1 -- 172.21.15.83:0/85417187 >> v1:172.21.15.83:6789/0 conn(0x7fd16c0b4bb0 legacy=0x7fd16c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd16959a700 1 -- 172.21.15.83:0/85417187 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822684332 0 0) 0x7fd16c06d4a0 con 0x7fd16c0b4bb0 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.200+0000 7fd16a59c700 1 -- 172.21.15.83:0/85417187 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c06c400 con 0x7fd16c15c890 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd16a59c700 1 --2- 172.21.15.83:0/85417187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c15c890 0x7fd16c0aee60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd160009820 tx=0x7fd160009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52b5d52ef4d299b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd16959a700 1 -- 172.21.15.83:0/85417187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160005e30 con 0x7fd16c15c890 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd16959a700 1 -- 172.21.15.83:0/85417187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160004030 con 0x7fd16c15c890 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd16959a700 1 -- 172.21.15.83:0/85417187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600043c0 con 0x7fd16c15c890 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 -- 172.21.15.83:0/85417187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c15c890 msgr2=0x7fd16c0aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 --2- 172.21.15.83:0/85417187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c15c890 0x7fd16c0aee60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd160009820 tx=0x7fd160009b30 comp rx=0 tx=0).stop 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 -- 172.21.15.83:0/85417187 shutdown_connections 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 --2- 172.21.15.83:0/85417187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c15c890 0x7fd16c0aee60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 -- 172.21.15.83:0/85417187 >> 172.21.15.83:0/85417187 conn(0x7fd16c0b2b60 msgr2=0x7fd16c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 -- 172.21.15.83:0/85417187 shutdown_connections 2023-11-28T11:11:10.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.201+0000 7fd170c6f700 1 -- 172.21.15.83:0/85417187 wait complete. 2023-11-28T11:11:10.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.203+0000 7fd170c6f700 1 Processor -- start 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.203+0000 7fd170c6f700 1 -- start start 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.203+0000 7fd170c6f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 0x7fd16c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.203+0000 7fd170c6f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd16c0b4bb0 0x7fd16c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.203+0000 7fd170c6f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd16c15c890 0x7fd16c0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd170c6f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c158e20 con 0x7fd16c06e370 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd170c6f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c158b40 con 0x7fd16c15c890 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd170c6f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c6e80 con 0x7fd16c0b4bb0 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 0x7fd16c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 0x7fd16c1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52240/0 (socket says 172.21.15.83:52240) 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 -- 172.21.15.83:0/3276316793 learned_addr learned my addr 172.21.15.83:0/3276316793 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 -- 172.21.15.83:0/3276316793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd16c0b4bb0 msgr2=0x7fd16c1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd16c0b4bb0 0x7fd16c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 -- 172.21.15.83:0/3276316793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd16c15c890 msgr2=0x7fd16c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd16c15c890 0x7fd16c0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 -- 172.21.15.83:0/3276316793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160009430 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd16a59c700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 0x7fd16c1582f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16001d630 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16001d790 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16001db40 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c0b5520 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.204+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c0b5240 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.205+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c0b5800 con 0x7fd16c06e370 2023-11-28T11:11:10.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.206+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd16002d070 con 0x7fd16c06e370 2023-11-28T11:11:10.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.206+0000 7fd15affd700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd14c0682f0 0x7fd14c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.206+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd160087940 con 0x7fd16c06e370 2023-11-28T11:11:10.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.206+0000 7fd169d9b700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd14c0682f0 0x7fd14c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.207+0000 7fd169d9b700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd14c0682f0 0x7fd14c06a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd16c06f940 tx=0x7fd15c0079f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.210+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd160053ff0 con 0x7fd16c06e370 2023-11-28T11:11:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.390+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd16c0d0bb0 con 0x7fd16c06e370 2023-11-28T11:11:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.391+0000 7fd15affd700 1 -- 172.21.15.83:0/3276316793 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd160057610 con 0x7fd16c06e370 2023-11-28T11:11:10.392 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd14c0682f0 msgr2=0x7fd14c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd14c0682f0 0x7fd14c06a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd16c06f940 tx=0x7fd15c0079f0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 msgr2=0x7fd16c1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 0x7fd16c1582f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd160004750 comp rx=0 tx=0).stop 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 shutdown_connections 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd16c15c890 0x7fd16c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd14c0682f0 0x7fd14c06a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd16c06e370 0x7fd16c1582f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 --2- 172.21.15.83:0/3276316793 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd16c0b4bb0 0x7fd16c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 >> 172.21.15.83:0/3276316793 conn(0x7fd16c0b2b60 msgr2=0x7fd16c0b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 shutdown_connections 2023-11-28T11:11:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.393+0000 7fd170c6f700 1 -- 172.21.15.83:0/3276316793 wait complete. 2023-11-28T11:11:10.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.524+0000 7fbfaba5f700 1 Processor -- start 2023-11-28T11:11:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.524+0000 7fbfaba5f700 1 -- start start 2023-11-28T11:11:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfaba5f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbfa40b88d0 0x7fbfa40cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfaba5f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfa4160e60 0x7fbfa40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfaba5f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40b5400 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfaba5f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa40b8ca0 con 0x7fbfa4160e60 2023-11-28T11:11:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfaba5f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa40d0c50 con 0x7fbfa40b88d0 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfa4160e60 0x7fbfa40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfa4160e60 0x7fbfa40b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52248/0 (socket says 172.21.15.83:52248) 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 -- 172.21.15.83:0/1947066825 learned_addr learned my addr 172.21.15.83:0/1947066825 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa97fb700 1 --2- 172.21.15.83:0/1947066825 >> v2:172.21.15.90:3300/0 conn(0x7fbfa40b88d0 0x7fbfa40cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 -- 172.21.15.83:0/1947066825 >> v2:172.21.15.90:3300/0 conn(0x7fbfa40b88d0 msgr2=0x7fbfa40cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 --2- 172.21.15.83:0/1947066825 >> v2:172.21.15.90:3300/0 conn(0x7fbfa40b88d0 0x7fbfa40cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 -- 172.21.15.83:0/1947066825 >> v1:172.21.15.90:6789/0 conn(0x7fbfa40b4fa0 legacy=0x7fbfa406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa8ffa700 1 -- 172.21.15.83:0/1947066825 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40cd760 con 0x7fbfa4160e60 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.525+0000 7fbfa97fb700 1 --2- 172.21.15.83:0/1947066825 >> v2:172.21.15.90:3300/0 conn(0x7fbfa40b88d0 0x7fbfa40cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfa8ffa700 1 --2- 172.21.15.83:0/1947066825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b4a60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbfa408bdc0 tx=0x7fbfa0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5894f0e152526725 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbf9bfff700 1 -- 172.21.15.83:0/1947066825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0004230 con 0x7fbfa4160e60 2023-11-28T11:11:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbf9bfff700 1 -- 172.21.15.83:0/1947066825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa0004390 con 0x7fbfa4160e60 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbf9bfff700 1 -- 172.21.15.83:0/1947066825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa00036a0 con 0x7fbfa4160e60 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfaba5f700 1 -- 172.21.15.83:0/1947066825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa4160e60 msgr2=0x7fbfa40b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/1947066825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b4a60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbfa408bdc0 tx=0x7fbfa0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfaba5f700 1 -- 172.21.15.83:0/1947066825 shutdown_connections 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/1947066825 >> v2:172.21.15.90:3300/0 conn(0x7fbfa40b88d0 0x7fbfa40cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/1947066825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b4a60 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.526+0000 7fbfaba5f700 1 -- 172.21.15.83:0/1947066825 >> 172.21.15.83:0/1947066825 conn(0x7fbfa406d030 msgr2=0x7fbfa40d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.527+0000 7fbfaba5f700 1 -- 172.21.15.83:0/1947066825 shutdown_connections 2023-11-28T11:11:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.527+0000 7fbfaba5f700 1 -- 172.21.15.83:0/1947066825 wait complete. 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 Processor -- start 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 -- start start 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 0x7fbfa40cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfa40b88d0 0x7fbfa40c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40c6510 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40cc550 con 0x7fbfa4160e60 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.528+0000 7fbfaba5f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40cafe0 con 0x7fbfa40b88d0 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 0x7fbfa40cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 0x7fbfa40cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52252/0 (socket says 172.21.15.83:52252) 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 -- 172.21.15.83:0/2803122312 learned_addr learned my addr 172.21.15.83:0/2803122312 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 -- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfa40b88d0 msgr2=0x7fbfa40c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa8ffa700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfa40b88d0 0x7fbfa40c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa9ffc700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfa40b88d0 0x7fbfa40c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 -- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfa4160e60 msgr2=0x7fbfa40b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 -- 172.21.15.83:0/2803122312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0009550 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfa97fb700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 0x7fbfa40cc9e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400c820 tx=0x7fbf9400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94011800 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94011960 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa40d0ef0 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94011d10 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.529+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa40cda40 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.530+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa41612d0 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.531+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf94021070 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.531+0000 7fbf99ffb700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf880682f0 0x7fbf8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.531+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbf94085cb0 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.531+0000 7fbfa8ffa700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf880682f0 0x7fbf8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.531+0000 7fbfa8ffa700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf880682f0 0x7fbf8806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfa41f5990 tx=0x7fbfa0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.535+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf94052350 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.758+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfa40cc830 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.759+0000 7fbf99ffb700 1 -- 172.21.15.83:0/2803122312 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fbf94055970 con 0x7fbfa40b4fa0 2023-11-28T11:11:10.760 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:10.760 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:10.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf880682f0 msgr2=0x7fbf8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf880682f0 0x7fbf8806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfa41f5990 tx=0x7fbfa0008030 comp rx=0 tx=0).stop 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 msgr2=0x7fbfa40cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 0x7fbfa40cc9e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400c820 tx=0x7fbf9400cbe0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 shutdown_connections 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfa4160e60 0x7fbfa40b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbf880682f0 0x7fbf8806a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfa40b4fa0 0x7fbfa40cc9e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 --2- 172.21.15.83:0/2803122312 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbfa40b88d0 0x7fbfa40c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 >> 172.21.15.83:0/2803122312 conn(0x7fbfa406d030 msgr2=0x7fbfa40cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 shutdown_connections 2023-11-28T11:11:10.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.761+0000 7fbfaba5f700 1 -- 172.21.15.83:0/2803122312 wait complete. 2023-11-28T11:11:10.765 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:11:10.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:10.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.893+0000 7f469d135700 1 Processor -- start 2023-11-28T11:11:10.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.893+0000 7f469d135700 1 -- start start 2023-11-28T11:11:10.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469d135700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f469815a510 0x7f46980cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469d135700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f46980b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469d135700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469815f1d0 con 0x7f46980cfdf0 2023-11-28T11:11:10.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469d135700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46980cd780 con 0x7f469815a510 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469d135700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469815eef0 con 0x7f469815a8e0 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f46980cfdf0 0x7f46980b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50120/0 (socket says 172.21.15.83:50120) 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.894+0000 7f469659c700 1 -- 172.21.15.83:0/4276031410 learned_addr learned my addr 172.21.15.83:0/4276031410 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811264821 0 0) 0x7f469815f1d0 con 0x7f46980cfdf0 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46800035b0 con 0x7f46980cfdf0 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4696d9d700 1 --2- 172.21.15.83:0/4276031410 >> v2:172.21.15.90:3300/0 conn(0x7f469815a510 0x7f46980cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469759e700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f46980b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911072442 0 0) 0x7f46800035b0 con 0x7f46980cfdf0 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 msgr2=0x7f46980b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f46980b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 >> v2:172.21.15.90:3300/0 conn(0x7f469815a510 msgr2=0x7f46980cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 --2- 172.21.15.83:0/4276031410 >> v2:172.21.15.90:3300/0 conn(0x7f469815a510 0x7f46980cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980cedd0 con 0x7f46980cfdf0 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f4688001670 con 0x7f46980cfdf0 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469759e700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f46980b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 >> v1:172.21.15.83:6789/0 conn(0x7f46980cfdf0 legacy=0x7f46980b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4680003a00 0x7f4680003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46800048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f4680005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469815eef0 con 0x7f4680004430 2023-11-28T11:11:10.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468000ba10 con 0x7f4680003a00 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468000bb50 con 0x7f469815a8e0 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46800048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f4696d9d700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4680003a00 0x7f4680003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469759e700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f4680005030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 -- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 msgr2=0x7f4680005030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f4680005030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 -- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4680003a00 msgr2=0x7f4680003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4680003a00 0x7f4680003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 -- 172.21.15.83:0/4276031410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800035b0 con 0x7f4680004430 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469759e700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f469815a8e0 0x7f4680005030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:10.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.895+0000 7f469659c700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46800048f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f468800c350 tx=0x7f4680005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2353cc6e1807c93 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468800ce50 con 0x7f4680004430 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688005960 con 0x7f4680004430 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f4695d9b700 1 -- 172.21.15.83:0/4276031410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688005d10 con 0x7f4680004430 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469d135700 1 -- 172.21.15.83:0/4276031410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 msgr2=0x7f46800048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469d135700 1 --2- 172.21.15.83:0/4276031410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46800048f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f468800c350 tx=0x7f4680005880 comp rx=0 tx=0).stop 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469759e700 1 -- 172.21.15.83:0/4276031410 reap_dead start 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469d135700 1 -- 172.21.15.83:0/4276031410 shutdown_connections 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469d135700 1 -- 172.21.15.83:0/4276031410 >> 172.21.15.83:0/4276031410 conn(0x7f46980cd960 msgr2=0x7f46980cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469d135700 1 -- 172.21.15.83:0/4276031410 shutdown_connections 2023-11-28T11:11:10.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.896+0000 7f469d135700 1 -- 172.21.15.83:0/4276031410 wait complete. 2023-11-28T11:11:10.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 Processor -- start 2023-11-28T11:11:10.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 -- start start 2023-11-28T11:11:10.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46981efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f469814eb00 0x7f469814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4698155ff0 0x7f4698156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806bc80 con 0x7f4680004430 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469814f4b0 con 0x7f469814eb00 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f469d135700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4698156d70 con 0x7f4698155ff0 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f4696d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46981efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f4696d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46981efba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52272/0 (socket says 172.21.15.83:52272) 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.898+0000 7f4696d9d700 1 -- 172.21.15.83:0/2710755205 learned_addr learned my addr 172.21.15.83:0/2710755205 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f469759e700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4698155ff0 0x7f4698156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f469659c700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f469814eb00 0x7f469814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f4696d9d700 1 -- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4698155ff0 msgr2=0x7f4698156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f4696d9d700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4698155ff0 0x7f4698156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f4696d9d700 1 -- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f469814eb00 msgr2=0x7f469814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f4696d9d700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f469814eb00 0x7f469814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f4696d9d700 1 -- 172.21.15.83:0/2710755205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980cf390 con 0x7f4680004430 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f4696d9d700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46981efba0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f468c011aa0 tx=0x7f468c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00f9e0 con 0x7f4680004430 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c00fb40 con 0x7f4680004430 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46980cf0b0 con 0x7f4680004430 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c014440 con 0x7f4680004430 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.899+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46980b3070 con 0x7f4680004430 2023-11-28T11:11:10.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.900+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698074320 con 0x7f4680004430 2023-11-28T11:11:10.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.900+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f468c0145a0 con 0x7f4680004430 2023-11-28T11:11:10.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.901+0000 7f467f7fe700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46840682f0 0x7f468406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.901+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f468c086750 con 0x7f4680004430 2023-11-28T11:11:10.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.901+0000 7f469659c700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46840682f0 0x7f468406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.901+0000 7f469659c700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46840682f0 0x7f468406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4688008fd0 tx=0x7f468800a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:10.905+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f468c052e70 con 0x7f4680004430 2023-11-28T11:11:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.087+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f469806f950 con 0x7f4680004430 2023-11-28T11:11:11.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.088+0000 7f467f7fe700 1 -- 172.21.15.83:0/2710755205 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f468c056490 con 0x7f4680004430 2023-11-28T11:11:11.090 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:11.090 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:27.384310+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:11.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46840682f0 msgr2=0x7f468406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46840682f0 0x7f468406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4688008fd0 tx=0x7f468800a000 comp rx=0 tx=0).stop 2023-11-28T11:11:11.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 msgr2=0x7f46981efba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46981efba0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f468c011aa0 tx=0x7f468c011e60 comp rx=0 tx=0).stop 2023-11-28T11:11:11.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 shutdown_connections 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4698155ff0 0x7f4698156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46840682f0 0x7f468406a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4680004430 0x7f46981efba0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 --2- 172.21.15.83:0/2710755205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f469814eb00 0x7f469814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 >> 172.21.15.83:0/2710755205 conn(0x7f46980cd960 msgr2=0x7f46980b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 shutdown_connections 2023-11-28T11:11:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.090+0000 7f469d135700 1 -- 172.21.15.83:0/2710755205 wait complete. 2023-11-28T11:11:11.104 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:11:11.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:11.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.222+0000 7fcf0792c700 1 Processor -- start 2023-11-28T11:11:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.222+0000 7fcf0792c700 1 -- start start 2023-11-28T11:11:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf0792c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf000b1fb0 0x7fcf000ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf0792c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000cbfb0 con 0x7fcf0015cd10 2023-11-28T11:11:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf0792c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000cb710 con 0x7fcf0015d0e0 2023-11-28T11:11:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf0792c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf000cb9f0 con 0x7fcf000b1fb0 2023-11-28T11:11:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf05ec9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf000b1fb0 0x7fcf000ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf056c8700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcf0015cd10 0x7fcf000b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50140/0 (socket says 172.21.15.83:50140) 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf056c8700 1 -- 172.21.15.83:0/1425701328 learned_addr learned my addr 172.21.15.83:0/1425701328 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf05ec9700 1 -- 172.21.15.83:0/1425701328 >> v1:172.21.15.90:6789/0 conn(0x7fcf0015d0e0 legacy=0x7fcf0006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf05ec9700 1 -- 172.21.15.83:0/1425701328 >> v1:172.21.15.83:6789/0 conn(0x7fcf0015cd10 legacy=0x7fcf000b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcf05ec9700 1 -- 172.21.15.83:0/1425701328 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000cbcd0 con 0x7fcf000b1fb0 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.223+0000 7fcef7fff700 1 -- 172.21.15.83:0/1425701328 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300978194 0 0) 0x7fcf000cbfb0 con 0x7fcf0015cd10 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf05ec9700 1 --2- 172.21.15.83:0/1425701328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf000b1fb0 0x7fcf000ca970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c7a8cc1f17ccb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcef7fff700 1 -- 172.21.15.83:0/1425701328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef800e070 con 0x7fcf000b1fb0 2023-11-28T11:11:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcef7fff700 1 -- 172.21.15.83:0/1425701328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef800bb90 con 0x7fcf000b1fb0 2023-11-28T11:11:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcef7fff700 1 -- 172.21.15.83:0/1425701328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8003a20 con 0x7fcf000b1fb0 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 -- 172.21.15.83:0/1425701328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf000b1fb0 msgr2=0x7fcf000ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 --2- 172.21.15.83:0/1425701328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf000b1fb0 0x7fcf000ca970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 -- 172.21.15.83:0/1425701328 shutdown_connections 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 --2- 172.21.15.83:0/1425701328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf000b1fb0 0x7fcf000ca970 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 -- 172.21.15.83:0/1425701328 >> 172.21.15.83:0/1425701328 conn(0x7fcf0006e9a0 msgr2=0x7fcf000cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 -- 172.21.15.83:0/1425701328 shutdown_connections 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.224+0000 7fcf0792c700 1 -- 172.21.15.83:0/1425701328 wait complete. 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.226+0000 7fcf0792c700 1 Processor -- start 2023-11-28T11:11:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.226+0000 7fcf0792c700 1 -- start start 2023-11-28T11:11:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf0792c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf000b1fb0 0x7fcf0015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf0792c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf0015cd10 0x7fcf0015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf0792c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 0x7fcf000adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf0792c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015e830 con 0x7fcf0015d0e0 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf0792c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b5ab0 con 0x7fcf0015cd10 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf0792c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015dbb0 con 0x7fcf000b1fb0 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 0x7fcf000adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 0x7fcf000adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52290/0 (socket says 172.21.15.83:52290) 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 -- 172.21.15.83:0/2218310524 learned_addr learned my addr 172.21.15.83:0/2218310524 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf04ec7700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf0015cd10 0x7fcf0015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf056c8700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf000b1fb0 0x7fcf0015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 -- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf000b1fb0 msgr2=0x7fcf0015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf000b1fb0 0x7fcf0015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 -- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf0015cd10 msgr2=0x7fcf0015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf0015cd10 0x7fcf0015acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 -- 172.21.15.83:0/2218310524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8009a70 con 0x7fcf0015d0e0 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf04ec7700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf0015cd10 0x7fcf0015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.227+0000 7fcf05ec9700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 0x7fcf000adbc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcf0006dc40 tx=0x7fcef8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.228+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012630 con 0x7fcf0015d0e0 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.228+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8012790 con 0x7fcf0015d0e0 2023-11-28T11:11:11.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.228+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012b40 con 0x7fcf0015d0e0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.228+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0006b660 con 0x7fcf0015d0e0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.228+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0006b9b0 con 0x7fcf0015d0e0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.229+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf000cd9f0 con 0x7fcf0015d0e0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.229+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcef802c070 con 0x7fcf0015d0e0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.229+0000 7fcef5ffb700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcee40682f0 0x7fcee406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.230+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcef8087f90 con 0x7fcf0015d0e0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.230+0000 7fcf056c8700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcee40682f0 0x7fcee406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.230+0000 7fcf056c8700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcee40682f0 0x7fcee406a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcef0007fd0 tx=0x7fcef0007c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.234+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcef8054650 con 0x7fcf0015d0e0 2023-11-28T11:11:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.456+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf00074320 con 0x7fcf0015d0e0 2023-11-28T11:11:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.457+0000 7fcef5ffb700 1 -- 172.21.15.83:0/2218310524 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fcef802a090 con 0x7fcf0015d0e0 2023-11-28T11:11:11.458 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:11.458 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.458+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcee40682f0 msgr2=0x7fcee406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.458+0000 7fcf0792c700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcee40682f0 0x7fcee406a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcef0007fd0 tx=0x7fcef0007c20 comp rx=0 tx=0).stop 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.458+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 msgr2=0x7fcf000adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.458+0000 7fcf0792c700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 0x7fcf000adbc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcf0006dc40 tx=0x7fcef8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 shutdown_connections 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fcee40682f0 0x7fcee406a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf0015d0e0 0x7fcf000adbc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcf000b1fb0 0x7fcf0015d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 --2- 172.21.15.83:0/2218310524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf0015cd10 0x7fcf0015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 >> 172.21.15.83:0/2218310524 conn(0x7fcf0006e9a0 msgr2=0x7fcf000ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 shutdown_connections 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.459+0000 7fcf0792c700 1 -- 172.21.15.83:0/2218310524 wait complete. 2023-11-28T11:11:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:11:11.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:11.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.590+0000 7fa8f03b9700 1 Processor -- start 2023-11-28T11:11:11.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.590+0000 7fa8f03b9700 1 -- start start 2023-11-28T11:11:11.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8f03b9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8f03b9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e815d5e0 con 0x7fa8e80cb0e0 2023-11-28T11:11:11.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8f03b9700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e8071530 con 0x7fa8e806d500 2023-11-28T11:11:11.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8f03b9700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e8071810 con 0x7fa8e8158ba0 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8ee956700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa8e80cb0e0 0x7fa8e815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50156/0 (socket says 172.21.15.83:50156) 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8ee956700 1 -- 172.21.15.83:0/137327005 learned_addr learned my addr 172.21.15.83:0/137327005 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8ee155700 1 --2- 172.21.15.83:0/137327005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221056080 0 0) 0x7fa8e815d5e0 con 0x7fa8e80cb0e0 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d40035b0 con 0x7fa8e80cb0e0 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.591+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661059032 0 0) 0x7fa8e8071530 con 0x7fa8e806d500 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e815d5e0 con 0x7fa8e806d500 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983196154 0 0) 0x7fa8d40035b0 con 0x7fa8e80cb0e0 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ee155700 1 -- 172.21.15.83:0/137327005 >> v1:172.21.15.90:6789/0 conn(0x7fa8e806d500 legacy=0x7fa8e815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ee155700 1 -- 172.21.15.83:0/137327005 >> v1:172.21.15.83:6789/0 conn(0x7fa8e80cb0e0 legacy=0x7fa8e815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ee155700 1 -- 172.21.15.83:0/137327005 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8071af0 con 0x7fa8e8158ba0 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ee155700 1 --2- 172.21.15.83:0/137327005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80cd0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009870 tx=0x7fa8d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969761bafec81c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00019f0 con 0x7fa8e8158ba0 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8004030 con 0x7fa8e8158ba0 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8ed153700 1 -- 172.21.15.83:0/137327005 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80043a0 con 0x7fa8e8158ba0 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8f03b9700 1 -- 172.21.15.83:0/137327005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 msgr2=0x7fa8e80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.592+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/137327005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80cd0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009870 tx=0x7fa8d8009b80 comp rx=0 tx=0).stop 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.593+0000 7fa8f03b9700 1 -- 172.21.15.83:0/137327005 shutdown_connections 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.593+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/137327005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80cd0b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.593+0000 7fa8f03b9700 1 -- 172.21.15.83:0/137327005 >> 172.21.15.83:0/137327005 conn(0x7fa8e80cf4e0 msgr2=0x7fa8e80cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.593+0000 7fa8f03b9700 1 -- 172.21.15.83:0/137327005 shutdown_connections 2023-11-28T11:11:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.593+0000 7fa8f03b9700 1 -- 172.21.15.83:0/137327005 wait complete. 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.594+0000 7fa8f03b9700 1 Processor -- start 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.594+0000 7fa8f03b9700 1 -- start start 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.594+0000 7fa8f03b9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 0x7fa8e80c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.594+0000 7fa8f03b9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8e80cb0e0 0x7fa8e80b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.594+0000 7fa8f03b9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8f03b9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815a7c0 con 0x7fa8e806d500 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8f03b9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815b340 con 0x7fa8e80cb0e0 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8f03b9700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815aa10 con 0x7fa8e8158ba0 2023-11-28T11:11:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 0x7fa8e80c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 0x7fa8e80c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52298/0 (socket says 172.21.15.83:52298) 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 -- 172.21.15.83:0/22339580 learned_addr learned my addr 172.21.15.83:0/22339580 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 -- 172.21.15.83:0/22339580 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 msgr2=0x7fa8e80c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 -- 172.21.15.83:0/22339580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8e80cb0e0 msgr2=0x7fa8e80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8e80cb0e0 0x7fa8e80b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 -- 172.21.15.83:0/22339580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80094a0 con 0x7fa8e806d500 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8ee155700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 0x7fa8e80c7800 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8000c00 tx=0x7fa8d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8003c30 con 0x7fa8e806d500 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d801f070 con 0x7fa8e806d500 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e80720b0 con 0x7fa8e806d500 2023-11-28T11:11:11.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8011630 con 0x7fa8e806d500 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.595+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e8071dd0 con 0x7fa8e806d500 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.596+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e8158f70 con 0x7fa8e806d500 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.596+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa8d802a020 con 0x7fa8e806d500 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.597+0000 7fa8deffd700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8cc0682f0 0x7fa8cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.597+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa8d80872e0 con 0x7fa8e806d500 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.597+0000 7fa8ed954700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8cc0682f0 0x7fa8cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.597+0000 7fa8ed954700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8cc0682f0 0x7fa8cc06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40010e0 tx=0x7fa8e40079e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.601+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8d8053a00 con 0x7fa8e806d500 2023-11-28T11:11:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.824+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8e8074320 con 0x7fa8e806d500 2023-11-28T11:11:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.824+0000 7fa8deffd700 1 -- 172.21.15.83:0/22339580 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fa8d8057020 con 0x7fa8e806d500 2023-11-28T11:11:11.826 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:11.826 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:26.382170+0000","modified":"2023-11-28T11:10:29.393540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6835/3041933349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3041933349},{"type":"v1","addr":"172.21.15.90:6835","nonce":3041933349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8cc0682f0 msgr2=0x7fa8cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8cc0682f0 0x7fa8cc06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40010e0 tx=0x7fa8e40079e0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 msgr2=0x7fa8e80c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 0x7fa8e80c7800 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8000c00 tx=0x7fa8d8004750 comp rx=0 tx=0).stop 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 shutdown_connections 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8e8158ba0 0x7fa8e80c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8cc0682f0 0x7fa8cc06a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8e806d500 0x7fa8e80c7800 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 --2- 172.21.15.83:0/22339580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8e80cb0e0 0x7fa8e80b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 >> 172.21.15.83:0/22339580 conn(0x7fa8e80cf4e0 msgr2=0x7fa8e80cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 shutdown_connections 2023-11-28T11:11:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.826+0000 7fa8f03b9700 1 -- 172.21.15.83:0/22339580 wait complete. 2023-11-28T11:11:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2023-11-28T11:11:11.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:11.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.959+0000 7fe846d72700 1 Processor -- start 2023-11-28T11:11:11.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe846d72700 1 -- start start 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe846d72700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe84015a480 0x7fe8400cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe846d72700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 0x7fe8400b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe846d72700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84015efa0 con 0x7fe8400cfce0 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe846d72700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400ceab0 con 0x7fe84015a480 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe846d72700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400cecc0 con 0x7fe84015a850 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe84530f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8400cfce0 0x7fe8400b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50170/0 (socket says 172.21.15.83:50170) 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.960+0000 7fe84530f700 1 -- 172.21.15.83:0/3741867197 learned_addr learned my addr 172.21.15.83:0/3741867197 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201452293 0 0) 0x7fe84015efa0 con 0x7fe8400cfce0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe844b0e700 1 --2- 172.21.15.83:0/3741867197 >> v2:172.21.15.90:3300/0 conn(0x7fe84015a480 0x7fe8400cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8300035b0 con 0x7fe8400cfce0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83ffff700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 0x7fe8400b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445554583 0 0) 0x7fe8300035b0 con 0x7fe8400cfce0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 msgr2=0x7fe8400b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 0x7fe8400b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 >> v2:172.21.15.90:3300/0 conn(0x7fe84015a480 msgr2=0x7fe8400cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 --2- 172.21.15.83:0/3741867197 >> v2:172.21.15.90:3300/0 conn(0x7fe84015a480 0x7fe8400cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400cefa0 con 0x7fe8400cfce0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fe8380019f0 con 0x7fe8400cfce0 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 >> v1:172.21.15.83:6789/0 conn(0x7fe8400cfce0 legacy=0x7fe8400b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe830003a00 0x7fe830003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 0x7fe830004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe830004d90 0x7fe830005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400ceab0 con 0x7fe830004d90 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400cecc0 con 0x7fe830003a00 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83000ba10 con 0x7fe84015a850 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe84530f700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe830004d90 0x7fe830005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe844b0e700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe830003a00 0x7fe830003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe83ffff700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 0x7fe830004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe84530f700 1 -- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 msgr2=0x7fe830004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.961+0000 7fe84530f700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe84015a850 0x7fe830004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe84530f700 1 -- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe830003a00 msgr2=0x7fe830003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe84530f700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe830003a00 0x7fe830003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe84530f700 1 -- 172.21.15.83:0/3741867197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300035b0 con 0x7fe830004d90 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe84530f700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe830004d90 0x7fe830005200 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe83800a8d0 tx=0x7fe830005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e78cf35a12b87b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800a340 con 0x7fe830004d90 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838012600 con 0x7fe830004d90 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe83f7fe700 1 -- 172.21.15.83:0/3741867197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8380129b0 con 0x7fe830004d90 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe846d72700 1 -- 172.21.15.83:0/3741867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe830004d90 msgr2=0x7fe830005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe846d72700 1 --2- 172.21.15.83:0/3741867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe830004d90 0x7fe830005200 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe83800a8d0 tx=0x7fe830005880 comp rx=0 tx=0).stop 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe84530f700 1 -- 172.21.15.83:0/3741867197 reap_dead start 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe846d72700 1 -- 172.21.15.83:0/3741867197 shutdown_connections 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe846d72700 1 -- 172.21.15.83:0/3741867197 >> 172.21.15.83:0/3741867197 conn(0x7fe84006de20 msgr2=0x7fe84006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe846d72700 1 -- 172.21.15.83:0/3741867197 shutdown_connections 2023-11-28T11:11:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.962+0000 7fe846d72700 1 -- 172.21.15.83:0/3741867197 wait complete. 2023-11-28T11:11:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.963+0000 7fe846d72700 1 Processor -- start 2023-11-28T11:11:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 -- start start 2023-11-28T11:11:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 0x7fe840151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840147540 0x7fe8400c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8400b5620 0x7fe8400b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b5e70 con 0x7fe840147170 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b6150 con 0x7fe840147540 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe846d72700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400c7fb0 con 0x7fe8400b5620 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe844b0e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 0x7fe840151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe844b0e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 0x7fe840151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52318/0 (socket says 172.21.15.83:52318) 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.964+0000 7fe844b0e700 1 -- 172.21.15.83:0/3577989165 learned_addr learned my addr 172.21.15.83:0/3577989165 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe83ffff700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840147540 0x7fe8400c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe84530f700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8400b5620 0x7fe8400b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe844b0e700 1 -- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8400b5620 msgr2=0x7fe8400b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe844b0e700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8400b5620 0x7fe8400b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe844b0e700 1 -- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840147540 msgr2=0x7fe8400c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe844b0e700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840147540 0x7fe8400c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe844b0e700 1 -- 172.21.15.83:0/3577989165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400b3070 con 0x7fe840147170 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe84530f700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8400b5620 0x7fe8400b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe844b0e700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 0x7fe840151320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe834011aa0 tx=0x7fe834011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400f9e0 con 0x7fe840147170 2023-11-28T11:11:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400fb40 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8400cf280 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834014440 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.965+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8400b3200 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.966+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840074320 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.966+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe8340145a0 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.967+0000 7fe83d7fa700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8240682f0 0x7fe82406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.967+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe834086750 con 0x7fe840147170 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.967+0000 7fe83ffff700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8240682f0 0x7fe82406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.967+0000 7fe83ffff700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8240682f0 0x7fe82406a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009ba0 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:11.971+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe834052e70 con 0x7fe840147170 2023-11-28T11:11:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.154+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe84006fe00 con 0x7fe840147170 2023-11-28T11:11:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.155+0000 7fe83d7fa700 1 -- 172.21.15.83:0/3577989165 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18074 (secure 0 0 0) 0x7fe834056490 con 0x7fe840147170 2023-11-28T11:11:12.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:12.157 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:10:27.384310+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:24.134118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:25.340097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8240682f0 msgr2=0x7fe82406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8240682f0 0x7fe82406a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009ba0 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 msgr2=0x7fe840151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 0x7fe840151320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe834011aa0 tx=0x7fe834011e60 comp rx=0 tx=0).stop 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 shutdown_connections 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8400b5620 0x7fe8400b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe8240682f0 0x7fe82406a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe840147170 0x7fe840151320 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 --2- 172.21.15.83:0/3577989165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840147540 0x7fe8400c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 >> 172.21.15.83:0/3577989165 conn(0x7fe84006de20 msgr2=0x7fe840157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:12.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 shutdown_connections 2023-11-28T11:11:12.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.157+0000 7fe846d72700 1 -- 172.21.15.83:0/3577989165 wait complete. 2023-11-28T11:11:12.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:11:12.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.289+0000 7fe724a26700 1 Processor -- start 2023-11-28T11:11:12.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.289+0000 7fe724a26700 1 -- start start 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe724a26700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe72015e5c0 0x7fe72015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe724a26700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe7200d11f0 0x7fe7200ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe724a26700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe720151a30 con 0x7fe72015b3b0 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe724a26700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200cb460 con 0x7fe7200d11f0 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe724a26700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200cb740 con 0x7fe72015e5c0 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe71dd9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe72015b3b0 0x7fe7200d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50186/0 (socket says 172.21.15.83:50186) 2023-11-28T11:11:12.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe71dd9b700 1 -- 172.21.15.83:0/2304721612 learned_addr learned my addr 172.21.15.83:0/2304721612 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe71ed9d700 1 --2- 172.21.15.83:0/2304721612 >> v2:172.21.15.90:3300/0 conn(0x7fe7200d11f0 0x7fe7200ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe71e59c700 1 --2- 172.21.15.83:0/2304721612 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe72015e5c0 0x7fe72015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe71d59a700 1 -- 172.21.15.83:0/2304721612 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849562946 0 0) 0x7fe720151a30 con 0x7fe72015b3b0 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.290+0000 7fe71d59a700 1 -- 172.21.15.83:0/2304721612 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7080035b0 con 0x7fe72015b3b0 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71d59a700 1 -- 172.21.15.83:0/2304721612 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185408606 0 0) 0x7fe7080035b0 con 0x7fe72015b3b0 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71e59c700 1 -- 172.21.15.83:0/2304721612 >> v2:172.21.15.90:3300/0 conn(0x7fe7200d11f0 msgr2=0x7fe7200ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71e59c700 1 --2- 172.21.15.83:0/2304721612 >> v2:172.21.15.90:3300/0 conn(0x7fe7200d11f0 0x7fe7200ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71e59c700 1 -- 172.21.15.83:0/2304721612 >> v1:172.21.15.83:6789/0 conn(0x7fe72015b3b0 legacy=0x7fe7200d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71e59c700 1 -- 172.21.15.83:0/2304721612 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72015abb0 con 0x7fe72015e5c0 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71ed9d700 1 --2- 172.21.15.83:0/2304721612 >> v2:172.21.15.90:3300/0 conn(0x7fe7200d11f0 0x7fe7200ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:12.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71e59c700 1 --2- 172.21.15.83:0/2304721612 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe72015e5c0 0x7fe72015e990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe714009840 tx=0x7fe714009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd180ad2f67fe925 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71d59a700 1 -- 172.21.15.83:0/2304721612 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710001670 con 0x7fe72015e5c0 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71d59a700 1 -- 172.21.15.83:0/2304721612 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714004030 con 0x7fe72015e5c0 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.291+0000 7fe71d59a700 1 -- 172.21.15.83:0/2304721612 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7140043c0 con 0x7fe72015e5c0 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 -- 172.21.15.83:0/2304721612 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe72015e5c0 msgr2=0x7fe72015e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 --2- 172.21.15.83:0/2304721612 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe72015e5c0 0x7fe72015e990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe714009840 tx=0x7fe714009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 -- 172.21.15.83:0/2304721612 shutdown_connections 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 --2- 172.21.15.83:0/2304721612 >> v2:172.21.15.90:3300/0 conn(0x7fe7200d11f0 0x7fe7200ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 --2- 172.21.15.83:0/2304721612 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe72015e5c0 0x7fe72015e990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 -- 172.21.15.83:0/2304721612 >> 172.21.15.83:0/2304721612 conn(0x7fe72006ea50 msgr2=0x7fe720070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 -- 172.21.15.83:0/2304721612 shutdown_connections 2023-11-28T11:11:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.292+0000 7fe724a26700 1 -- 172.21.15.83:0/2304721612 wait complete. 2023-11-28T11:11:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.293+0000 7fe724a26700 1 Processor -- start 2023-11-28T11:11:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.293+0000 7fe724a26700 1 -- start start 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe724a26700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 0x7fe72014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe724a26700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe72015e5c0 0x7fe7200c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe724a26700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe7200c4d50 0x7fe72015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe724a26700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200b4d80 con 0x7fe72015b3b0 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe724a26700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72014b1f0 con 0x7fe72015e5c0 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe724a26700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200c5360 con 0x7fe7200c4d50 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 0x7fe72014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 0x7fe72014acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52332/0 (socket says 172.21.15.83:52332) 2023-11-28T11:11:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 -- 172.21.15.83:0/1966835405 learned_addr learned my addr 172.21.15.83:0/1966835405 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71dd9b700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe72015e5c0 0x7fe7200c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71ed9d700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe7200c4d50 0x7fe72015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 -- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe7200c4d50 msgr2=0x7fe72015b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe7200c4d50 0x7fe72015b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 -- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe72015e5c0 msgr2=0x7fe7200c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe72015e5c0 0x7fe7200c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 -- 172.21.15.83:0/1966835405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718009a70 con 0x7fe72015b3b0 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71ed9d700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe7200c4d50 0x7fe72015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.294+0000 7fe71e59c700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 0x7fe72014acb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe7140090b0 tx=0x7fe714004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:12.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.295+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714003d70 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.295+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71401f070 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.295+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714011870 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.295+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7140094a0 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.295+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72015eed0 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.296+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72015ae90 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.296+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe71402d070 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.296+0000 7fe70effd700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe7000682f0 0x7fe70006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.296+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe714021070 con 0x7fe72015b3b0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.296+0000 7fe71dd9b700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe7000682f0 0x7fe70006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.297+0000 7fe71dd9b700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe7000682f0 0x7fe70006a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7201f35f0 tx=0x7fe710001230 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.301+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe714053b20 con 0x7fe72015b3b0 2023-11-28T11:11:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.524+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7200cba20 con 0x7fe72015b3b0 2023-11-28T11:11:12.889 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-28T11:11:12.888+0000 7fdf5c1f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:11:12.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.891+0000 7fe70effd700 1 -- 172.21.15.83:0/1966835405 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe714057140 con 0x7fe72015b3b0 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe7000682f0 msgr2=0x7fe70006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe7000682f0 0x7fe70006a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7201f35f0 tx=0x7fe710001230 comp rx=0 tx=0).stop 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 msgr2=0x7fe72014acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 0x7fe72014acb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe7140090b0 tx=0x7fe714004750 comp rx=0 tx=0).stop 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 shutdown_connections 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe7200c4d50 0x7fe72015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe7000682f0 0x7fe70006a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe72015b3b0 0x7fe72014acb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 --2- 172.21.15.83:0/1966835405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe72015e5c0 0x7fe7200c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 >> 172.21.15.83:0/1966835405 conn(0x7fe7200d11f0 msgr2=0x7fe72015c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 shutdown_connections 2023-11-28T11:11:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:12.893+0000 7fe724a26700 1 -- 172.21.15.83:0/1966835405 wait complete. 2023-11-28T11:11:12.896 INFO:tasks.ceph.mds.d.smithi090.stderr: -7670> 2023-11-28T11:10:21.020+0000 7fc284311700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-28T11:11:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:11:12.897 INFO:tasks.ceph.mds.d.smithi090.stderr: -6465> 2023-11-28T11:10:31.957+0000 7fc27e305700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:11:12.902 INFO:tasks.ceph.mds.d.smithi090.stderr: -2957> 2023-11-28T11:10:34.785+0000 7fc27aafe700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5225=0-4194304@1} caps={5225=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5225 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b2fc5efb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:11:12.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:11:12.930 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2023-11-28T11:11:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.029+0000 7f75d8547700 1 Processor -- start 2023-11-28T11:11:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.029+0000 7f75d8547700 1 -- start start 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d8547700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75d00af220 0x7f75d015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d8547700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f75d015d8b0 0x7f75d006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d8547700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d015dee0 con 0x7f75d00aee50 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d8547700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d015dcf0 con 0x7f75d00af220 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d8547700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d00cf950 con 0x7f75d015d8b0 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75d00af220 0x7f75d015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75d00af220 0x7f75d015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52340/0 (socket says 172.21.15.83:52340) 2023-11-28T11:11:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d62e3700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f75d00aee50 0x7f75d00d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50198/0 (socket says 172.21.15.83:50198) 2023-11-28T11:11:13.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 -- 172.21.15.83:0/310269732 learned_addr learned my addr 172.21.15.83:0/310269732 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:13.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d6ae4700 1 --2- 172.21.15.83:0/310269732 >> v2:172.21.15.90:3300/0 conn(0x7f75d015d8b0 0x7f75d006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 -- 172.21.15.83:0/310269732 >> v2:172.21.15.90:3300/0 conn(0x7f75d015d8b0 msgr2=0x7f75d006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 --2- 172.21.15.83:0/310269732 >> v2:172.21.15.90:3300/0 conn(0x7f75d015d8b0 0x7f75d006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 -- 172.21.15.83:0/310269732 >> v1:172.21.15.83:6789/0 conn(0x7f75d00aee50 legacy=0x7f75d00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d5ae2700 1 -- 172.21.15.83:0/310269732 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00cfc30 con 0x7f75d00af220 2023-11-28T11:11:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d52e1700 1 -- 172.21.15.83:0/310269732 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357208769 0 0) 0x7f75d015dee0 con 0x7f75d00aee50 2023-11-28T11:11:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.030+0000 7f75d6ae4700 1 --2- 172.21.15.83:0/310269732 >> v2:172.21.15.90:3300/0 conn(0x7f75d015d8b0 0x7f75d006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d5ae2700 1 --2- 172.21.15.83:0/310269732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d015d370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f75d006d970 tx=0x7f75cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afadb0b829a5fe94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d52e1700 1 -- 172.21.15.83:0/310269732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00b870 con 0x7f75d00af220 2023-11-28T11:11:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d52e1700 1 -- 172.21.15.83:0/310269732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc00b9d0 con 0x7f75d00af220 2023-11-28T11:11:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d52e1700 1 -- 172.21.15.83:0/310269732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00bd60 con 0x7f75d00af220 2023-11-28T11:11:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 -- 172.21.15.83:0/310269732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 msgr2=0x7f75d015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 --2- 172.21.15.83:0/310269732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d015d370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f75d006d970 tx=0x7f75cc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 -- 172.21.15.83:0/310269732 shutdown_connections 2023-11-28T11:11:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 --2- 172.21.15.83:0/310269732 >> v2:172.21.15.90:3300/0 conn(0x7f75d015d8b0 0x7f75d006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 --2- 172.21.15.83:0/310269732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d015d370 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 -- 172.21.15.83:0/310269732 >> 172.21.15.83:0/310269732 conn(0x7f75d00b2b30 msgr2=0x7f75d00b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 -- 172.21.15.83:0/310269732 shutdown_connections 2023-11-28T11:11:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.031+0000 7f75d8547700 1 -- 172.21.15.83:0/310269732 wait complete. 2023-11-28T11:11:13.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.032+0000 7f75d8547700 1 Processor -- start 2023-11-28T11:11:13.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 -- start start 2023-11-28T11:11:13.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d00aee50 0x7f75d009d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d009bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75d015d8b0 0x7f75d0099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d014eb80 con 0x7f75d00af220 2023-11-28T11:11:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d009c000 con 0x7f75d00aee50 2023-11-28T11:11:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d8547700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d009c2c0 con 0x7f75d015d8b0 2023-11-28T11:11:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d5ae2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d009bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d5ae2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d009bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52346/0 (socket says 172.21.15.83:52346) 2023-11-28T11:11:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d5ae2700 1 -- 172.21.15.83:0/2275519404 learned_addr learned my addr 172.21.15.83:0/2275519404 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d62e3700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d00aee50 0x7f75d009d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.033+0000 7f75d6ae4700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75d015d8b0 0x7f75d0099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d5ae2700 1 -- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75d015d8b0 msgr2=0x7f75d0099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d5ae2700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75d015d8b0 0x7f75d0099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d5ae2700 1 -- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d00aee50 msgr2=0x7f75d009d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d5ae2700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d00aee50 0x7f75d009d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d5ae2700 1 -- 172.21.15.83:0/2275519404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc009550 con 0x7f75d00af220 2023-11-28T11:11:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d5ae2700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d009bac0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f75cc014040 tx=0x7f75cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc004220 con 0x7f75d00af220 2023-11-28T11:11:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc01e070 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc004580 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d00b1910 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.034+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d00cff10 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.035+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d015e1c0 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.035+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75cc00f050 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.036+0000 7f75c6ffd700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75b40682f0 0x7f75b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.036+0000 7f75d62e3700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75b40682f0 0x7f75b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.036+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f75cc020070 con 0x7f75d00af220 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.036+0000 7f75d62e3700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75b40682f0 0x7f75b406a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f75c0000f60 tx=0x7f75c0007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.040+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75cc0535e0 con 0x7f75d00af220 2023-11-28T11:11:13.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.267+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f75d00742b0 con 0x7f75d00af220 2023-11-28T11:11:13.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.901+0000 7f75c6ffd700 1 -- 172.21.15.83:0/2275519404 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75cc0150a0 con 0x7f75d00af220 2023-11-28T11:11:13.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.903+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75b40682f0 msgr2=0x7f75b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.903+0000 7f75d8547700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75b40682f0 0x7f75b406a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f75c0000f60 tx=0x7f75c0007e70 comp rx=0 tx=0).stop 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.903+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 msgr2=0x7f75d009bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.903+0000 7f75d8547700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d009bac0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f75cc014040 tx=0x7f75cc004750 comp rx=0 tx=0).stop 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 shutdown_connections 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f75d015d8b0 0x7f75d0099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d00aee50 0x7f75d009d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75b40682f0 0x7f75b406a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 --2- 172.21.15.83:0/2275519404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d00af220 0x7f75d009bac0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 >> 172.21.15.83:0/2275519404 conn(0x7f75d00b2b30 msgr2=0x7f75d006e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 shutdown_connections 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:13.904+0000 7f75d8547700 1 -- 172.21.15.83:0/2275519404 wait complete. 2023-11-28T11:11:13.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:11:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.047+0000 7fa86cf4a700 1 Processor -- start 2023-11-28T11:11:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.047+0000 7fa86cf4a700 1 -- start start 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86cf4a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa8680b2230 0x7fa86806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86cf4a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8680b5050 0x7fa8680d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86cf4a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8680b2600 con 0x7fa8680d2760 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86cf4a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680cdb20 con 0x7fa8680b5050 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86cf4a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680b5cf0 con 0x7fa8680b2230 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8680b5050 0x7fa8680d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8680b5050 0x7fa8680d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52356/0 (socket says 172.21.15.83:52356) 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 -- 172.21.15.83:0/2301086263 learned_addr learned my addr 172.21.15.83:0/2301086263 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86659c700 1 --2- 172.21.15.83:0/2301086263 >> v2:172.21.15.90:3300/0 conn(0x7fa8680b2230 0x7fa86806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 -- 172.21.15.83:0/2301086263 >> v2:172.21.15.90:3300/0 conn(0x7fa8680b2230 msgr2=0x7fa86806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 --2- 172.21.15.83:0/2301086263 >> v2:172.21.15.90:3300/0 conn(0x7fa8680b2230 0x7fa86806ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 -- 172.21.15.83:0/2301086263 >> v1:172.21.15.90:6789/0 conn(0x7fa8680d2760 legacy=0x7fa8680b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 -- 172.21.15.83:0/2301086263 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680cd480 con 0x7fa8680b5050 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa866d9d700 1 --2- 172.21.15.83:0/2301086263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b5050 0x7fa8680d0c40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa8680b56e0 tx=0x7fa86000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9ad2997e85cbdd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.048+0000 7fa86559a700 1 -- 172.21.15.83:0/2301086263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000e070 con 0x7fa8680b5050 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86559a700 1 -- 172.21.15.83:0/2301086263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860003a20 con 0x7fa8680b5050 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86559a700 1 -- 172.21.15.83:0/2301086263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860003dd0 con 0x7fa8680b5050 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 -- 172.21.15.83:0/2301086263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b5050 msgr2=0x7fa8680d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/2301086263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b5050 0x7fa8680d0c40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa8680b56e0 tx=0x7fa86000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 -- 172.21.15.83:0/2301086263 shutdown_connections 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/2301086263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b5050 0x7fa8680d0c40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/2301086263 >> v2:172.21.15.90:3300/0 conn(0x7fa8680b2230 0x7fa86806ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 -- 172.21.15.83:0/2301086263 >> 172.21.15.83:0/2301086263 conn(0x7fa8680d30a0 msgr2=0x7fa8680d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 -- 172.21.15.83:0/2301086263 shutdown_connections 2023-11-28T11:11:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.049+0000 7fa86cf4a700 1 -- 172.21.15.83:0/2301086263 wait complete. 2023-11-28T11:11:14.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 Processor -- start 2023-11-28T11:11:14.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 -- start start 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 0x7fa8681540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8680b5050 0x7fa8680ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8680d2760 0x7fa8680d1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa868153370 con 0x7fa8680b2230 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8681f64d0 con 0x7fa8680b5050 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86cf4a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8681535c0 con 0x7fa8680d2760 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 0x7fa8681540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 0x7fa8681540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52358/0 (socket says 172.21.15.83:52358) 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa86659c700 1 -- 172.21.15.83:0/183429678 learned_addr learned my addr 172.21.15.83:0/183429678 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa866d9d700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8680d2760 0x7fa8680d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.051+0000 7fa865d9b700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8680b5050 0x7fa8680ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86659c700 1 -- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8680d2760 msgr2=0x7fa8680d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86659c700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8680d2760 0x7fa8680d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86659c700 1 -- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8680b5050 msgr2=0x7fa8680ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86659c700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8680b5050 0x7fa8680ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86659c700 1 -- 172.21.15.83:0/183429678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009b30 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa865d9b700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8680b5050 0x7fa8680ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86659c700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 0x7fa8681540a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa85800e810 tx=0x7fa85800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858013610 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858013770 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8680cf360 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858013b20 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.052+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8680cf080 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.053+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8680cf640 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.053+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa858023070 con 0x7fa8680b2230 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.054+0000 7fa856ffd700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8480682f0 0x7fa84806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.054+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fa858085a90 con 0x7fa8680b2230 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.054+0000 7fa865d9b700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8480682f0 0x7fa84806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.054+0000 7fa865d9b700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8480682f0 0x7fa84806a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa8681f5650 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.058+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8580520d0 con 0x7fa8680b2230 2023-11-28T11:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.239+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8680742b0 con 0x7fa8680b2230 2023-11-28T11:11:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.954+0000 7fa856ffd700 1 -- 172.21.15.83:0/183429678 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa8580556f0 con 0x7fa8680b2230 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8480682f0 msgr2=0x7fa84806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8480682f0 0x7fa84806a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa8681f5650 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 msgr2=0x7fa8681540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 0x7fa8681540a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa85800e810 tx=0x7fa85800eb20 comp rx=0 tx=0).stop 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 shutdown_connections 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8680d2760 0x7fa8680d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa8480682f0 0x7fa84806a7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b2230 0x7fa8681540a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 --2- 172.21.15.83:0/183429678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8680b5050 0x7fa8680ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 >> 172.21.15.83:0/183429678 conn(0x7fa8680d30a0 msgr2=0x7fa86815ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 shutdown_connections 2023-11-28T11:11:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:14.957+0000 7fa86cf4a700 1 -- 172.21.15.83:0/183429678 wait complete. 2023-11-28T11:11:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:11:14.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:11:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.098+0000 7f794968d700 1 Processor -- start 2023-11-28T11:11:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.098+0000 7f794968d700 1 -- start start 2023-11-28T11:11:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f794968d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f794415ed70 0x7f79440cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f794968d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79440cb3b0 0x7f79440cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f794968d700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79440cbc50 con 0x7f79440aff70 2023-11-28T11:11:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f794968d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794415e8e0 con 0x7f794415ed70 2023-11-28T11:11:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f794968d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79440cb970 con 0x7f79440cb3b0 2023-11-28T11:11:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f79437fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79440cb3b0 0x7f79440cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f79427fc700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f794415ed70 0x7f79440cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f7942ffd700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f79440aff70 0x7f794415b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:48572/0 (socket says 172.21.15.83:48572) 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f7942ffd700 1 -- 172.21.15.83:0/1154548113 learned_addr learned my addr 172.21.15.83:0/1154548113 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f79427fc700 1 -- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79440cb3b0 msgr2=0x7f79440cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.099+0000 7f79427fc700 1 --2- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79440cb3b0 0x7f79440cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f79427fc700 1 -- 172.21.15.83:0/1154548113 >> v1:172.21.15.90:6789/0 conn(0x7f79440aff70 legacy=0x7f794415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f79427fc700 1 -- 172.21.15.83:0/1154548113 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794415e290 con 0x7f794415ed70 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f7941ffb700 1 -- 172.21.15.83:0/1154548113 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879632421 0 0) 0x7f7934001110 con 0x7f79440aff70 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f79427fc700 1 --2- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f794415ed70 0x7f79440cd740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7938009840 tx=0x7f7938009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e2f4fceef9b323 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f7941ffb700 1 -- 172.21.15.83:0/1154548113 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938011830 con 0x7f794415ed70 2023-11-28T11:11:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f7941ffb700 1 -- 172.21.15.83:0/1154548113 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938011990 con 0x7f794415ed70 2023-11-28T11:11:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f7941ffb700 1 -- 172.21.15.83:0/1154548113 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938011d20 con 0x7f794415ed70 2023-11-28T11:11:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f794968d700 1 -- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f794415ed70 msgr2=0x7f79440cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f794968d700 1 --2- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f794415ed70 0x7f79440cd740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7938009840 tx=0x7f7938009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f794968d700 1 -- 172.21.15.83:0/1154548113 shutdown_connections 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f794968d700 1 --2- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79440cb3b0 0x7f79440cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f794968d700 1 --2- 172.21.15.83:0/1154548113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f794415ed70 0x7f79440cd740 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.100+0000 7f794968d700 1 -- 172.21.15.83:0/1154548113 >> 172.21.15.83:0/1154548113 conn(0x7f794406f300 msgr2=0x7f794406f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.101+0000 7f794968d700 1 -- 172.21.15.83:0/1154548113 shutdown_connections 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.101+0000 7f794968d700 1 -- 172.21.15.83:0/1154548113 wait complete. 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.102+0000 7f794968d700 1 Processor -- start 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.102+0000 7f794968d700 1 -- start start 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.102+0000 7f794968d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 0x7f79440cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.102+0000 7f794968d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79440cb3b0 0x7f79440b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f794968d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f794415ed70 0x7f79440b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f794968d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79440af950 con 0x7f79440aff70 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f794968d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79440ce200 con 0x7f79440cb3b0 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f794968d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79440b5a00 con 0x7f794415ed70 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 0x7f79440cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 0x7f79440cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52372/0 (socket says 172.21.15.83:52372) 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 -- 172.21.15.83:0/2914091103 learned_addr learned my addr 172.21.15.83:0/2914091103 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f79427fc700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79440cb3b0 0x7f79440b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f79437fe700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f794415ed70 0x7f79440b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 -- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f794415ed70 msgr2=0x7f79440b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f794415ed70 0x7f79440b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 -- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79440cb3b0 msgr2=0x7f79440b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79440cb3b0 0x7f79440b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 -- 172.21.15.83:0/2914091103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380094a0 con 0x7f79440aff70 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f7942ffd700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 0x7f79440cdcc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f793400c730 tx=0x7f793400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934004c80 con 0x7f79440aff70 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934004e00 con 0x7f79440aff70 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79440b0340 con 0x7f79440aff70 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934007310 con 0x7f79440aff70 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.103+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794415e570 con 0x7f79440aff70 2023-11-28T11:11:15.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.104+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79440ce930 con 0x7f79440aff70 2023-11-28T11:11:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.105+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f793401a020 con 0x7f79440aff70 2023-11-28T11:11:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.105+0000 7f792b7fe700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79300682f0 0x7f793006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.105+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f7934085790 con 0x7f79440aff70 2023-11-28T11:11:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.105+0000 7f79427fc700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79300682f0 0x7f793006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.106+0000 7f79427fc700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79300682f0 0x7f793006a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7938000c00 tx=0x7f7938008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.109+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7934051f90 con 0x7f79440aff70 2023-11-28T11:11:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.291+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79440cbc50 con 0x7f79440aff70 2023-11-28T11:11:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.967+0000 7f792b7fe700 1 -- 172.21.15.83:0/2914091103 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f79440cbc50 con 0x7f79440aff70 2023-11-28T11:11:15.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79300682f0 msgr2=0x7f793006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:15.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79300682f0 0x7f793006a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7938000c00 tx=0x7f7938008030 comp rx=0 tx=0).stop 2023-11-28T11:11:15.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 msgr2=0x7f79440cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:15.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 0x7f79440cdcc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f793400c730 tx=0x7f793400caf0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 shutdown_connections 2023-11-28T11:11:15.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f794415ed70 0x7f79440b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79300682f0 0x7f793006a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79440aff70 0x7f79440cdcc0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 --2- 172.21.15.83:0/2914091103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79440cb3b0 0x7f79440b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:15.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 >> 172.21.15.83:0/2914091103 conn(0x7f794406f300 msgr2=0x7f794415a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:15.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 shutdown_connections 2023-11-28T11:11:15.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:15.971+0000 7f794968d700 1 -- 172.21.15.83:0/2914091103 wait complete. 2023-11-28T11:11:15.975 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-28T11:11:15.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:11:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.113+0000 7f4465f99700 1 Processor -- start 2023-11-28T11:11:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.113+0000 7f4465f99700 1 -- start start 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f4465f99700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44600af1f0 0x7f44600d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f4465f99700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446006c800 con 0x7f446006c370 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f4465f99700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446006cae0 con 0x7f44600aee20 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f4465f99700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446006cdc0 con 0x7f44600af1f0 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44600af1f0 0x7f44600d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44600af1f0 0x7f44600d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52384/0 (socket says 172.21.15.83:52384) 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445ffff700 1 -- 172.21.15.83:0/1007713856 learned_addr learned my addr 172.21.15.83:0/1007713856 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445effd700 1 --1- 172.21.15.83:0/1007713856 >> v1:172.21.15.83:6789/0 conn(0x7f446006c370 0x7f44600cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50240/0 (socket says 172.21.15.83:50240) 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445ffff700 1 -- 172.21.15.83:0/1007713856 >> v1:172.21.15.90:6789/0 conn(0x7f44600aee20 legacy=0x7f446006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445ffff700 1 -- 172.21.15.83:0/1007713856 >> v1:172.21.15.83:6789/0 conn(0x7f446006c370 legacy=0x7f44600cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445ffff700 1 -- 172.21.15.83:0/1007713856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446006d0a0 con 0x7f44600af1f0 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.114+0000 7f445e7fc700 1 -- 172.21.15.83:0/1007713856 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272167118 0 0) 0x7f446006c800 con 0x7f446006c370 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f445ffff700 1 --2- 172.21.15.83:0/1007713856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44600af1f0 0x7f44600d0710 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4458009e50 tx=0x7f445800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd24e2688c99f00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f445e7fc700 1 -- 172.21.15.83:0/1007713856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800e070 con 0x7f44600af1f0 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f445e7fc700 1 -- 172.21.15.83:0/1007713856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458003a20 con 0x7f44600af1f0 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f445e7fc700 1 -- 172.21.15.83:0/1007713856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458003dd0 con 0x7f44600af1f0 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 -- 172.21.15.83:0/1007713856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44600af1f0 msgr2=0x7f44600d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 --2- 172.21.15.83:0/1007713856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44600af1f0 0x7f44600d0710 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4458009e50 tx=0x7f445800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 -- 172.21.15.83:0/1007713856 shutdown_connections 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 --2- 172.21.15.83:0/1007713856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44600af1f0 0x7f44600d0710 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 -- 172.21.15.83:0/1007713856 >> 172.21.15.83:0/1007713856 conn(0x7f44600b2b60 msgr2=0x7f44600b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 -- 172.21.15.83:0/1007713856 shutdown_connections 2023-11-28T11:11:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.115+0000 7f4465f99700 1 -- 172.21.15.83:0/1007713856 wait complete. 2023-11-28T11:11:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.117+0000 7f4465f99700 1 Processor -- start 2023-11-28T11:11:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.117+0000 7f4465f99700 1 -- start start 2023-11-28T11:11:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.117+0000 7f4465f99700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 0x7f44601ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.117+0000 7f4465f99700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44600aee20 0x7f446015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.117+0000 7f4465f99700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44600af1f0 0x7f446015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.117+0000 7f4465f99700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4460156540 con 0x7f446006c370 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4465f99700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4460155930 con 0x7f44600aee20 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4465f99700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4460155c10 con 0x7f44600af1f0 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 0x7f44601ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 0x7f44601ee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52386/0 (socket says 172.21.15.83:52386) 2023-11-28T11:11:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 -- 172.21.15.83:0/2764297001 learned_addr learned my addr 172.21.15.83:0/2764297001 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445effd700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44600aee20 0x7f446015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445ffff700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44600af1f0 0x7f446015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 -- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44600af1f0 msgr2=0x7f446015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44600af1f0 0x7f446015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 -- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44600aee20 msgr2=0x7f446015c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44600aee20 0x7f446015c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 -- 172.21.15.83:0/2764297001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4458009b30 con 0x7f446006c370 2023-11-28T11:11:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f445f7fe700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 0x7f44601ee4c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f445400c4b0 tx=0x7f445400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454011790 con 0x7f446006c370 2023-11-28T11:11:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44540118f0 con 0x7f446006c370 2023-11-28T11:11:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446006d380 con 0x7f446006c370 2023-11-28T11:11:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454011ca0 con 0x7f446006c370 2023-11-28T11:11:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.118+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446006fc70 con 0x7f446006c370 2023-11-28T11:11:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.119+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446006f490 con 0x7f446006c370 2023-11-28T11:11:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.120+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4454010050 con 0x7f446006c370 2023-11-28T11:11:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.120+0000 7f4447fff700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f444c0682f0 0x7f444c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.120+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4454085540 con 0x7f446006c370 2023-11-28T11:11:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.120+0000 7f445effd700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f444c0682f0 0x7f444c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.120+0000 7f445effd700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f444c0682f0 0x7f444c06a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f446015bd90 tx=0x7f4450007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.124+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4454051f00 con 0x7f446006c370 2023-11-28T11:11:16.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.305+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f44600b3820 con 0x7f446006c370 2023-11-28T11:11:16.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.309+0000 7f4447fff700 1 -- 172.21.15.83:0/2764297001 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4454055520 con 0x7f446006c370 2023-11-28T11:11:16.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f444c0682f0 msgr2=0x7f444c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f444c0682f0 0x7f444c06a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f446015bd90 tx=0x7f4450007c00 comp rx=0 tx=0).stop 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 msgr2=0x7f44601ee4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 0x7f44601ee4c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f445400c4b0 tx=0x7f445400c870 comp rx=0 tx=0).stop 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 shutdown_connections 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f44600af1f0 0x7f446015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f444c0682f0 0x7f444c06a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f446006c370 0x7f44601ee4c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 --2- 172.21.15.83:0/2764297001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44600aee20 0x7f446015c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 >> 172.21.15.83:0/2764297001 conn(0x7f44600b2b60 msgr2=0x7f44600b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 shutdown_connections 2023-11-28T11:11:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.311+0000 7f4465f99700 1 -- 172.21.15.83:0/2764297001 wait complete. 2023-11-28T11:11:16.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:11:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.443+0000 7f105e0b9700 1 Processor -- start 2023-11-28T11:11:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.443+0000 7f105e0b9700 1 -- start start 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f105e0b9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f105815cd10 0x7f10580ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f105e0b9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f10580b1fb0 0x7f10580cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f105e0b9700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10580cbfb0 con 0x7f105815d0e0 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f105e0b9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105806cee0 con 0x7f10580b1fb0 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f105e0b9700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580cb710 con 0x7f105815cd10 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f10580b1fb0 0x7f10580cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f10580b1fb0 0x7f10580cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52398/0 (socket says 172.21.15.83:52398) 2023-11-28T11:11:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 -- 172.21.15.83:0/124694176 learned_addr learned my addr 172.21.15.83:0/124694176 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f10577fe700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f105815cd10 0x7f10580ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 -- 172.21.15.83:0/124694176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f105815cd10 msgr2=0x7f10580ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f105815cd10 0x7f10580ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 -- 172.21.15.83:0/124694176 >> v1:172.21.15.90:6789/0 conn(0x7f105815d0e0 legacy=0x7f10580b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 -- 172.21.15.83:0/124694176 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580cb4e0 con 0x7f10580b1fb0 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f10577fe700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f105815cd10 0x7f10580ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f1057fff700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10580b1fb0 0x7f10580cafa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1050009e50 tx=0x7f105000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4168927f6b5d311 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f10567fc700 1 -- 172.21.15.83:0/124694176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000e070 con 0x7f10580b1fb0 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f10567fc700 1 -- 172.21.15.83:0/124694176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050003a20 con 0x7f10580b1fb0 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.444+0000 7f10567fc700 1 -- 172.21.15.83:0/124694176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050003dd0 con 0x7f10580b1fb0 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 -- 172.21.15.83:0/124694176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10580b1fb0 msgr2=0x7f10580cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10580b1fb0 0x7f10580cafa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1050009e50 tx=0x7f105000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 -- 172.21.15.83:0/124694176 shutdown_connections 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10580b1fb0 0x7f10580cafa0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 --2- 172.21.15.83:0/124694176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f105815cd10 0x7f10580ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 -- 172.21.15.83:0/124694176 >> 172.21.15.83:0/124694176 conn(0x7f105806e9a0 msgr2=0x7f10580cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 -- 172.21.15.83:0/124694176 shutdown_connections 2023-11-28T11:11:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.445+0000 7f105e0b9700 1 -- 172.21.15.83:0/124694176 wait complete. 2023-11-28T11:11:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.446+0000 7f105e0b9700 1 Processor -- start 2023-11-28T11:11:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.446+0000 7f105e0b9700 1 -- start start 2023-11-28T11:11:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f105e0b9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10580b1fb0 0x7f105814a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f105e0b9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105815cd10 0x7f10580b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f105e0b9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 0x7f10580b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f105e0b9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105814aba0 con 0x7f105815d0e0 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f105e0b9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105814a1d0 con 0x7f105815cd10 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f105e0b9700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580b39f0 con 0x7f10580b1fb0 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 0x7f10580b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 0x7f10580b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52404/0 (socket says 172.21.15.83:52404) 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 -- 172.21.15.83:0/594249222 learned_addr learned my addr 172.21.15.83:0/594249222 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1056ffd700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105815cd10 0x7f10580b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f10577fe700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10580b1fb0 0x7f105814a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 -- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10580b1fb0 msgr2=0x7f105814a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10580b1fb0 0x7f105814a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 -- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105815cd10 msgr2=0x7f10580b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105815cd10 0x7f10580b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1057fff700 1 -- 172.21.15.83:0/594249222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050009b30 con 0x7f105815d0e0 2023-11-28T11:11:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.447+0000 7f1056ffd700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105815cd10 0x7f10580b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.448+0000 7f1057fff700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 0x7f10580b4e40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f105806dc40 tx=0x7f1050004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.448+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050004440 con 0x7f105815d0e0 2023-11-28T11:11:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.448+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10500045a0 con 0x7f105815d0e0 2023-11-28T11:11:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.448+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050004950 con 0x7f105815d0e0 2023-11-28T11:11:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.448+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10580cbcd0 con 0x7f105815d0e0 2023-11-28T11:11:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.448+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10580cb9f0 con 0x7f105815d0e0 2023-11-28T11:11:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.449+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105806b9b0 con 0x7f105815d0e0 2023-11-28T11:11:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.449+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f105001c440 con 0x7f105815d0e0 2023-11-28T11:11:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.449+0000 7f103ffff700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10440682f0 0x7f104406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.449+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f10500871f0 con 0x7f105815d0e0 2023-11-28T11:11:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.449+0000 7f10577fe700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10440682f0 0x7f104406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.450+0000 7f10577fe700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10440682f0 0x7f104406a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1048006fd0 tx=0x7f1048008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.454+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1050053a60 con 0x7f105815d0e0 2023-11-28T11:11:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.707+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f105814a380 con 0x7f105815d0e0 2023-11-28T11:11:16.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.957+0000 7f103ffff700 1 -- 172.21.15.83:0/594249222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v152) v1 ==== 149+0+0 (secure 0 0 0) 0x7f105814a380 con 0x7f105815d0e0 2023-11-28T11:11:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10440682f0 msgr2=0x7f104406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10440682f0 0x7f104406a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1048006fd0 tx=0x7f1048008040 comp rx=0 tx=0).stop 2023-11-28T11:11:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 msgr2=0x7f10580b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 0x7f10580b4e40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f105806dc40 tx=0x7f1050004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 shutdown_connections 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f10440682f0 0x7f104406a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f105815d0e0 0x7f10580b4e40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10580b1fb0 0x7f105814a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 --2- 172.21.15.83:0/594249222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105815cd10 0x7f10580b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 >> 172.21.15.83:0/594249222 conn(0x7f105806e9a0 msgr2=0x7f10580afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 shutdown_connections 2023-11-28T11:11:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:16.959+0000 7f105e0b9700 1 -- 172.21.15.83:0/594249222 wait complete. 2023-11-28T11:11:16.973 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T11:11:16.973 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T11:11:16.973 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:11:16.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:11:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.093+0000 7f6c9aebd700 1 Processor -- start 2023-11-28T11:11:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.093+0000 7f6c9aebd700 1 -- start start 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c9aebd700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6c9415d0e0 0x7f6c940ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c9aebd700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c940cbfb0 con 0x7f6c940b1fb0 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c9aebd700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c940cb710 con 0x7f6c9415cd10 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c9aebd700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c940cb9f0 con 0x7f6c9415d0e0 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c9945a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6c940b1fb0 0x7f6c9406bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50272/0 (socket says 172.21.15.83:50272) 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c9945a700 1 -- 172.21.15.83:0/1505852118 learned_addr learned my addr 172.21.15.83:0/1505852118 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003241767 0 0) 0x7f6c940cbfb0 con 0x7f6c940b1fb0 2023-11-28T11:11:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c93fff700 1 --2- 172.21.15.83:0/1505852118 >> v2:172.21.15.90:3300/0 conn(0x7f6c9415d0e0 0x7f6c940ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.094+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c840035b0 con 0x7f6c940b1fb0 2023-11-28T11:11:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 85341661 0 0) 0x7f6c840035b0 con 0x7f6c940b1fb0 2023-11-28T11:11:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 >> v2:172.21.15.90:3300/0 conn(0x7f6c9415d0e0 msgr2=0x7f6c940ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 --2- 172.21.15.83:0/1505852118 >> v2:172.21.15.90:3300/0 conn(0x7f6c9415d0e0 0x7f6c940ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 >> v1:172.21.15.90:6789/0 conn(0x7f6c9415cd10 legacy=0x7f6c940b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940cbcd0 con 0x7f6c940b1fb0 2023-11-28T11:11:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6c8c0019f0 con 0x7f6c940b1fb0 2023-11-28T11:11:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 >> v1:172.21.15.83:6789/0 conn(0x7f6c940b1fb0 legacy=0x7f6c9406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c84003a00 0x7f6c84003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84004850 0x7f6c84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c84005200 0x7f6c8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940cb9f0 con 0x7f6c84005200 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8400c080 con 0x7f6c84003a00 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8400c240 con 0x7f6c84004850 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c84005200 0x7f6c8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c93fff700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84004850 0x7f6c84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c98c59700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c84003a00 0x7f6c84003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 -- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84004850 msgr2=0x7f6c84004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84004850 0x7f6c84004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 -- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c84003a00 msgr2=0x7f6c84003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c84003a00 0x7f6c84003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 -- 172.21.15.83:0/1505852118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c840035b0 con 0x7f6c84005200 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 -- 172.21.15.83:0/1505852118 reap_dead start 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c9945a700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c84005200 0x7f6c8400ba10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00a800 tx=0x7f6c84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a067e5b38a1bfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.095+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c00a2f0 con 0x7f6c84005200 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c012600 con 0x7f6c84005200 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c937fe700 1 -- 172.21.15.83:0/1505852118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0129b0 con 0x7f6c84005200 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 -- 172.21.15.83:0/1505852118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c84005200 msgr2=0x7f6c8400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c84005200 0x7f6c8400ba10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00a800 tx=0x7f6c84005880 comp rx=0 tx=0).stop 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 -- 172.21.15.83:0/1505852118 shutdown_connections 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/1505852118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c84005200 0x7f6c8400ba10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 -- 172.21.15.83:0/1505852118 >> 172.21.15.83:0/1505852118 conn(0x7f6c9406e9a0 msgr2=0x7f6c940cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:17.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 -- 172.21.15.83:0/1505852118 shutdown_connections 2023-11-28T11:11:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.096+0000 7f6c9aebd700 1 -- 172.21.15.83:0/1505852118 wait complete. 2023-11-28T11:11:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.097+0000 7f6c9aebd700 1 Processor -- start 2023-11-28T11:11:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.097+0000 7f6c9aebd700 1 -- start start 2023-11-28T11:11:17.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9aebd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84005200 0x7f6c94153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9aebd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c940c3df0 0x7f6c940c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9aebd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 0x7f6c940c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9aebd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c5ba0 con 0x7f6c940c47a0 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9aebd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94154480 con 0x7f6c940c3df0 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9aebd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9415fcc0 con 0x7f6c84005200 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9945a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 0x7f6c940c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9945a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 0x7f6c940c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52422/0 (socket says 172.21.15.83:52422) 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c9945a700 1 -- 172.21.15.83:0/570868292 learned_addr learned my addr 172.21.15.83:0/570868292 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c93fff700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c940c3df0 0x7f6c940c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.098+0000 7f6c98c59700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84005200 0x7f6c94153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9945a700 1 -- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84005200 msgr2=0x7f6c94153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9945a700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84005200 0x7f6c94153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9945a700 1 -- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c940c3df0 msgr2=0x7f6c940c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9945a700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c940c3df0 0x7f6c940c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9945a700 1 -- 172.21.15.83:0/570868292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9406b660 con 0x7f6c940c47a0 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c98c59700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84005200 0x7f6c94153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c93fff700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c940c3df0 0x7f6c940c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9945a700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 0x7f6c940c6ef0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6c94072970 tx=0x7f6c8c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0019f0 con 0x7f6c940c47a0 2023-11-28T11:11:17.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c01d420 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9406b9b0 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c01d810 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.099+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c940cd9f0 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.100+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94074320 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.101+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c8c010070 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.101+0000 7f6c917fa700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c780682f0 0x7f6c7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.101+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6c8c087b50 con 0x7f6c940c47a0 2023-11-28T11:11:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.101+0000 7f6c98c59700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c780682f0 0x7f6c7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:17.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.101+0000 7f6c98c59700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c780682f0 0x7f6c7806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c00 tx=0x7f6c88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:17.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.107+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c8c054570 con 0x7f6c940c47a0 2023-11-28T11:11:17.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.368+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f6c940b27b0 con 0x7f6c940c47a0 2023-11-28T11:11:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.961+0000 7f6c917fa700 1 -- 172.21.15.83:0/570868292 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v153) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6c8c057b90 con 0x7f6c940c47a0 2023-11-28T11:11:17.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c780682f0 msgr2=0x7f6c7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c780682f0 0x7f6c7806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c00 tx=0x7f6c88009040 comp rx=0 tx=0).stop 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 msgr2=0x7f6c940c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 0x7f6c940c6ef0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6c94072970 tx=0x7f6c8c001d80 comp rx=0 tx=0).stop 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 shutdown_connections 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c780682f0 0x7f6c7806a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c940c47a0 0x7f6c940c6ef0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c84005200 0x7f6c94153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 --2- 172.21.15.83:0/570868292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c940c3df0 0x7f6c940c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:17.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 >> 172.21.15.83:0/570868292 conn(0x7f6c9406e9a0 msgr2=0x7f6c9415bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:17.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 shutdown_connections 2023-11-28T11:11:17.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:17.963+0000 7f6c9aebd700 1 -- 172.21.15.83:0/570868292 wait complete. 2023-11-28T11:11:17.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:11:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.106+0000 7f97b5e94700 1 Processor -- start 2023-11-28T11:11:18.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.106+0000 7f97b5e94700 1 -- start start 2023-11-28T11:11:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97b5e94700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97b5e94700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97b015d8b0 0x7f97b006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97b5e94700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b015dee0 con 0x7f97b00aee50 2023-11-28T11:11:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97b5e94700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00cf950 con 0x7f97b015d8b0 2023-11-28T11:11:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97b5e94700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00cfc30 con 0x7f97b00af220 2023-11-28T11:11:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97b015d8b0 0x7f97b006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97b015d8b0 0x7f97b006c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52432/0 (socket says 172.21.15.83:52432) 2023-11-28T11:11:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 -- 172.21.15.83:0/2009234608 learned_addr learned my addr 172.21.15.83:0/2009234608 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:18.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97af7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f97b00aee50 0x7f97b00d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50288/0 (socket says 172.21.15.83:50288) 2023-11-28T11:11:18.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97aeffd700 1 --2- 172.21.15.83:0/2009234608 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 -- 172.21.15.83:0/2009234608 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 msgr2=0x7f97b015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:18.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 --2- 172.21.15.83:0/2009234608 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 -- 172.21.15.83:0/2009234608 >> v1:172.21.15.83:6789/0 conn(0x7f97b00aee50 legacy=0x7f97b00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 -- 172.21.15.83:0/2009234608 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00cff10 con 0x7f97b015d8b0 2023-11-28T11:11:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97ae7fc700 1 -- 172.21.15.83:0/2009234608 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796065027 0 0) 0x7f97b015dee0 con 0x7f97b00aee50 2023-11-28T11:11:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.107+0000 7f97affff700 1 --2- 172.21.15.83:0/2009234608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b006c600 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f97a00098f0 tx=0x7f97a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fd7d3ddc605af0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97ae7fc700 1 -- 172.21.15.83:0/2009234608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000b8a0 con 0x7f97b015d8b0 2023-11-28T11:11:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97ae7fc700 1 -- 172.21.15.83:0/2009234608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000ba00 con 0x7f97b015d8b0 2023-11-28T11:11:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97ae7fc700 1 -- 172.21.15.83:0/2009234608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000bd90 con 0x7f97b015d8b0 2023-11-28T11:11:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 -- 172.21.15.83:0/2009234608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 msgr2=0x7f97b006c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 --2- 172.21.15.83:0/2009234608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b006c600 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f97a00098f0 tx=0x7f97a0009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 -- 172.21.15.83:0/2009234608 shutdown_connections 2023-11-28T11:11:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 --2- 172.21.15.83:0/2009234608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b006c600 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 --2- 172.21.15.83:0/2009234608 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 -- 172.21.15.83:0/2009234608 >> 172.21.15.83:0/2009234608 conn(0x7f97b00b2b30 msgr2=0x7f97b00b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 -- 172.21.15.83:0/2009234608 shutdown_connections 2023-11-28T11:11:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.108+0000 7f97b5e94700 1 -- 172.21.15.83:0/2009234608 wait complete. 2023-11-28T11:11:18.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.109+0000 7f97b5e94700 1 Processor -- start 2023-11-28T11:11:18.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 -- start start 2023-11-28T11:11:18.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97b00aee50 0x7f97b0143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b01f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b00cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00c5ab0 con 0x7f97b015d8b0 2023-11-28T11:11:18.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00c54a0 con 0x7f97b00aee50 2023-11-28T11:11:18.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97b5e94700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00ce7f0 con 0x7f97b00af220 2023-11-28T11:11:18.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97affff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b00cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97affff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b00cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52438/0 (socket says 172.21.15.83:52438) 2023-11-28T11:11:18.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.110+0000 7f97affff700 1 -- 172.21.15.83:0/3789829276 learned_addr learned my addr 172.21.15.83:0/3789829276 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:18.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97af7fe700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97b00aee50 0x7f97b0143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97aeffd700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b01f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97affff700 1 -- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 msgr2=0x7f97b01f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97affff700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b01f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97affff700 1 -- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97b00aee50 msgr2=0x7f97b0143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97affff700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97b00aee50 0x7f97b0143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97affff700 1 -- 172.21.15.83:0/3789829276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009550 con 0x7f97b015d8b0 2023-11-28T11:11:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97aeffd700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b01f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97affff700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b00cd790 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f97a000b0e0 tx=0x7f97a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0004300 con 0x7f97b015d8b0 2023-11-28T11:11:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a001f070 con 0x7f97b015d8b0 2023-11-28T11:11:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b015e1c0 con 0x7f97b015d8b0 2023-11-28T11:11:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00036a0 con 0x7f97b015d8b0 2023-11-28T11:11:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.111+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b00b1910 con 0x7f97b015d8b0 2023-11-28T11:11:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.112+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b006cb40 con 0x7f97b015d8b0 2023-11-28T11:11:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.112+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97a002a020 con 0x7f97b015d8b0 2023-11-28T11:11:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.113+0000 7f97a7fff700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97900682f0 0x7f979006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.113+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f97a0086fd0 con 0x7f97b015d8b0 2023-11-28T11:11:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.113+0000 7f97af7fe700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97900682f0 0x7f979006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.113+0000 7f97af7fe700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97900682f0 0x7f979006a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f97a80012e0 tx=0x7f97a8001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.118+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97a00539f0 con 0x7f97b015d8b0 2023-11-28T11:11:18.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.299+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f97b0072350 con 0x7f97b015d8b0 2023-11-28T11:11:18.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.302+0000 7f97a7fff700 1 -- 172.21.15.83:0/3789829276 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f97a0057010 con 0x7f97b015d8b0 2023-11-28T11:11:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97900682f0 msgr2=0x7f979006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97900682f0 0x7f979006a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f97a80012e0 tx=0x7f97a8001aa0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 msgr2=0x7f97b00cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b00cd790 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f97a000b0e0 tx=0x7f97a0004750 comp rx=0 tx=0).stop 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 shutdown_connections 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97900682f0 0x7f979006a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97b015d8b0 0x7f97b00cd790 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97b00aee50 0x7f97b0143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 --2- 172.21.15.83:0/3789829276 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97b00af220 0x7f97b01f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 >> 172.21.15.83:0/3789829276 conn(0x7f97b00b2b30 msgr2=0x7f97b006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 shutdown_connections 2023-11-28T11:11:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.304+0000 7f97b5e94700 1 -- 172.21.15.83:0/3789829276 wait complete. 2023-11-28T11:11:18.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:11:18.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:18.369 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:18.416 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:11:18.417 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:11:18.417 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:18.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:11:18.434 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:11:18.434 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:18.456 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:18.456 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:11:18.456 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:18.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:11:18.491 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:11:18.492 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:18.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:18.514 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:11:18.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.650+0000 7f4e3b099700 1 Processor -- start 2023-11-28T11:11:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.650+0000 7f4e3b099700 1 -- start start 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e3b099700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e3415a3f0 0x7f4e340b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e3b099700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4e3415cff0 0x7f4e340cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e3b099700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e340cb1d0 con 0x7f4e340b1720 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e3b099700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e340cb540 con 0x7f4e3415a3f0 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e3b099700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e340b2030 con 0x7f4e3415cff0 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e33fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e3415a3f0 0x7f4e340b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e38e35700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4e340b1720 0x7f4e340cd870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50302/0 (socket says 172.21.15.83:50302) 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e38e35700 1 -- 172.21.15.83:0/1490155418 learned_addr learned my addr 172.21.15.83:0/1490155418 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e33fff700 1 -- 172.21.15.83:0/1490155418 >> v2:172.21.15.90:3300/0 conn(0x7f4e3415cff0 msgr2=0x7f4e340cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e39636700 1 --2- 172.21.15.83:0/1490155418 >> v2:172.21.15.90:3300/0 conn(0x7f4e3415cff0 0x7f4e340cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e33fff700 1 --2- 172.21.15.83:0/1490155418 >> v2:172.21.15.90:3300/0 conn(0x7f4e3415cff0 0x7f4e340cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e337fe700 1 -- 172.21.15.83:0/1490155418 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021810338 0 0) 0x7f4e340cb1d0 con 0x7f4e340b1720 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e33fff700 1 -- 172.21.15.83:0/1490155418 >> v1:172.21.15.83:6789/0 conn(0x7f4e340b1720 legacy=0x7f4e340cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.651+0000 7f4e33fff700 1 -- 172.21.15.83:0/1490155418 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3415d840 con 0x7f4e3415a3f0 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e33fff700 1 --2- 172.21.15.83:0/1490155418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415a3f0 0x7f4e340b1af0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4e3406bde0 tx=0x7f4e20009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e595f45a1d4591 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e337fe700 1 -- 172.21.15.83:0/1490155418 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011830 con 0x7f4e3415a3f0 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e337fe700 1 -- 172.21.15.83:0/1490155418 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20011990 con 0x7f4e3415a3f0 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e337fe700 1 -- 172.21.15.83:0/1490155418 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011d20 con 0x7f4e3415a3f0 2023-11-28T11:11:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 -- 172.21.15.83:0/1490155418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415a3f0 msgr2=0x7f4e340b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 --2- 172.21.15.83:0/1490155418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415a3f0 0x7f4e340b1af0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4e3406bde0 tx=0x7f4e20009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 -- 172.21.15.83:0/1490155418 shutdown_connections 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 --2- 172.21.15.83:0/1490155418 >> v2:172.21.15.90:3300/0 conn(0x7f4e3415cff0 0x7f4e340cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 --2- 172.21.15.83:0/1490155418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415a3f0 0x7f4e340b1af0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 -- 172.21.15.83:0/1490155418 >> 172.21.15.83:0/1490155418 conn(0x7f4e340cc720 msgr2=0x7f4e340ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 -- 172.21.15.83:0/1490155418 shutdown_connections 2023-11-28T11:11:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.652+0000 7f4e3b099700 1 -- 172.21.15.83:0/1490155418 wait complete. 2023-11-28T11:11:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.654+0000 7f4e3b099700 1 Processor -- start 2023-11-28T11:11:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.654+0000 7f4e3b099700 1 -- start start 2023-11-28T11:11:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e3b099700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e340b1720 0x7f4e3414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e3b099700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e3415a3f0 0x7f4e340c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e3b099700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 0x7f4e340ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e3b099700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3414e540 con 0x7f4e3415cff0 2023-11-28T11:11:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e3b099700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e340c6d70 con 0x7f4e3415a3f0 2023-11-28T11:11:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e3b099700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e340c7050 con 0x7f4e340b1720 2023-11-28T11:11:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 0x7f4e340ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 0x7f4e340ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52452/0 (socket says 172.21.15.83:52452) 2023-11-28T11:11:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 -- 172.21.15.83:0/916773393 learned_addr learned my addr 172.21.15.83:0/916773393 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e33fff700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e3415a3f0 0x7f4e340c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e38e35700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e340b1720 0x7f4e3414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 -- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e340b1720 msgr2=0x7f4e3414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e340b1720 0x7f4e3414e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 -- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e3415a3f0 msgr2=0x7f4e340c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e3415a3f0 0x7f4e340c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e39636700 1 -- 172.21.15.83:0/916773393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200094a0 con 0x7f4e3415cff0 2023-11-28T11:11:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.655+0000 7f4e33fff700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e3415a3f0 0x7f4e340c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.656+0000 7f4e39636700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 0x7f4e340ccd60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00cc30 tx=0x7f4e2c00a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.656+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c00ab70 con 0x7f4e3415cff0 2023-11-28T11:11:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.656+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c00acd0 con 0x7f4e3415cff0 2023-11-28T11:11:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.656+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3415de00 con 0x7f4e3415cff0 2023-11-28T11:11:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.656+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c003c40 con 0x7f4e3415cff0 2023-11-28T11:11:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.656+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3415db20 con 0x7f4e3415cff0 2023-11-28T11:11:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.657+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3415a870 con 0x7f4e3415cff0 2023-11-28T11:11:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.657+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e2c01f070 con 0x7f4e3415cff0 2023-11-28T11:11:18.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.658+0000 7f4e317fa700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e180682f0 0x7f4e1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:18.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.658+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4e2c085bb0 con 0x7f4e3415cff0 2023-11-28T11:11:18.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.658+0000 7f4e38e35700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e180682f0 0x7f4e1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:18.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.658+0000 7f4e38e35700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e180682f0 0x7f4e1806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4e28000f60 tx=0x7f4e28007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.663+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e2c0525d0 con 0x7f4e3415cff0 2023-11-28T11:11:18.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.892+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e34074320 con 0x7f4e3415cff0 2023-11-28T11:11:18.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.892+0000 7f4e317fa700 1 -- 172.21.15.83:0/916773393 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f4e2c055bf0 con 0x7f4e3415cff0 2023-11-28T11:11:18.893 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:18.893 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3432468322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":3432468322},{"type":"v1","addr":"172.21.15.90:6835","nonce":3432468322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1230735850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1230735850},{"type":"v1","addr":"172.21.15.83:6837","nonce":1230735850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2889501499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2889501499},{"type":"v1","addr":"172.21.15.83:6836","nonce":2889501499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/3174478043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3174478043},{"type":"v1","addr":"172.21.15.90:6834","nonce":3174478043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2023-11-28T11:11:18.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e180682f0 msgr2=0x7f4e1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e180682f0 0x7f4e1806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4e28000f60 tx=0x7f4e28007be0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 msgr2=0x7f4e340ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:18.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 0x7f4e340ccd60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00cc30 tx=0x7f4e2c00a1f0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 shutdown_connections 2023-11-28T11:11:18.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f4e180682f0 0x7f4e1806a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e3415cff0 0x7f4e340ccd60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4e340b1720 0x7f4e3414e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 --2- 172.21.15.83:0/916773393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e3415a3f0 0x7f4e340c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:18.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 >> 172.21.15.83:0/916773393 conn(0x7f4e340cc720 msgr2=0x7f4e340ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:18.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 shutdown_connections 2023-11-28T11:11:18.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:18.894+0000 7f4e3b099700 1 -- 172.21.15.83:0/916773393 wait complete. 2023-11-28T11:11:18.900 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2023-11-28T11:11:18.907 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:11:18.907 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:11:18.907 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:11:18.908 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:18.908 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:11:18.908 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:11:18.908 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:11:18.908 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:18.909 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:11:18.909 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:11:18.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:11:18.909 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:18.909 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:11:18.910 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:11:18.910 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:11:18.910 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:18.917 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:11:18.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:11:18.918 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:11:18.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:11:18.926 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:11:18.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:11:18.928 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:11:18.928 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:11:18.959 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:11:18.960 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:18.960 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:11:18.960 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:18.969 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:11:18.969 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:18.970 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:11:18.970 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:18.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:11:19.040 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-28T11:11:19.041 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-28T11:11:19.055 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2023-11-28T11:11:19.056 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2023-11-28T11:11:19.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.101+0000 7f0dfd193700 1 Processor -- start 2023-11-28T11:11:19.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0dfd193700 1 -- start start 2023-11-28T11:11:19.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0dfd193700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df80b87e0 0x7f0df80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:19.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0dfd193700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0df80b4ff0 0x7f0df80d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:19.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0dfd193700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df80b26e0 con 0x7f0df8160bc0 2023-11-28T11:11:19.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0dfd193700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df80d0a30 con 0x7f0df80b4ff0 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0dfd193700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80d0d10 con 0x7f0df80b87e0 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0df759e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0df80b4ff0 0x7f0df80d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0df759e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0df80b4ff0 0x7f0df80d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52482/0 (socket says 172.21.15.83:52482) 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.102+0000 7f0df759e700 1 -- 172.21.15.83:0/1443893989 learned_addr learned my addr 172.21.15.83:0/1443893989 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df6d9d700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df80b87e0 0x7f0df80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df759e700 1 -- 172.21.15.83:0/1443893989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df80b87e0 msgr2=0x7f0df80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df759e700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df80b87e0 0x7f0df80cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df759e700 1 -- 172.21.15.83:0/1443893989 >> v1:172.21.15.90:6789/0 conn(0x7f0df8160bc0 legacy=0x7f0df80b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df759e700 1 -- 172.21.15.83:0/1443893989 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df8161030 con 0x7f0df80b4ff0 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df6d9d700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df80b87e0 0x7f0df80cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:19.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df759e700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df80b4ff0 0x7f0df80d11a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0de80098f0 tx=0x7f0de8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8403d98a2170d2d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df5d9b700 1 -- 172.21.15.83:0/1443893989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800b8a0 con 0x7f0df80b4ff0 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df5d9b700 1 -- 172.21.15.83:0/1443893989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de800ba00 con 0x7f0df80b4ff0 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0df5d9b700 1 -- 172.21.15.83:0/1443893989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800bd90 con 0x7f0df80b4ff0 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0dfd193700 1 -- 172.21.15.83:0/1443893989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df80b4ff0 msgr2=0x7f0df80d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0dfd193700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df80b4ff0 0x7f0df80d11a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0de80098f0 tx=0x7f0de8009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0dfd193700 1 -- 172.21.15.83:0/1443893989 shutdown_connections 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0dfd193700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df80b4ff0 0x7f0df80d11a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:19.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0dfd193700 1 --2- 172.21.15.83:0/1443893989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df80b87e0 0x7f0df80cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.103+0000 7f0dfd193700 1 -- 172.21.15.83:0/1443893989 >> 172.21.15.83:0/1443893989 conn(0x7f0df806cef0 msgr2=0x7f0df80cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.104+0000 7f0dfd193700 1 -- 172.21.15.83:0/1443893989 shutdown_connections 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.104+0000 7f0dfd193700 1 -- 172.21.15.83:0/1443893989 wait complete. 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.105+0000 7f0dfd193700 1 Processor -- start 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.105+0000 7f0dfd193700 1 -- start start 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0df80b87e0 0x7f0df815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df8160bc0 0x7f0df814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 0x7f0df80c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df814d850 con 0x7f0df815db60 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df815e040 con 0x7f0df80b87e0 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df815e320 con 0x7f0df8160bc0 2023-11-28T11:11:19.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 0x7f0df80c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 0x7f0df80c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52488/0 (socket says 172.21.15.83:52488) 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 -- 172.21.15.83:0/2615127292 learned_addr learned my addr 172.21.15.83:0/2615127292 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df6d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0df80b87e0 0x7f0df815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df659c700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df8160bc0 0x7f0df814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 -- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df8160bc0 msgr2=0x7f0df814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df8160bc0 0x7f0df814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 -- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0df80b87e0 msgr2=0x7f0df815f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0df80b87e0 0x7f0df815f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 -- 172.21.15.83:0/2615127292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8009550 con 0x7f0df815db60 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df6d9d700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0df80b87e0 0x7f0df815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0df759e700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 0x7f0df80c9f30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009130 tx=0x7f0de8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:19.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8004510 con 0x7f0df815db60 2023-11-28T11:11:19.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de801e070 con 0x7f0df815db60 2023-11-28T11:11:19.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df80cd750 con 0x7f0df815db60 2023-11-28T11:11:19.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.106+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de80038e0 con 0x7f0df815db60 2023-11-28T11:11:19.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.107+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df80cd4b0 con 0x7f0df815db60 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.107+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df80cda30 con 0x7f0df815db60 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.108+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0de800f050 con 0x7f0df815db60 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.108+0000 7f0def7fe700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0dd80682f0 0x7f0dd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.108+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0de8020070 con 0x7f0df815db60 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.108+0000 7f0df6d9d700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0dd80682f0 0x7f0dd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.109+0000 7f0df6d9d700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0dd80682f0 0x7f0dd806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0df0003ef0 tx=0x7f0df00097f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:19.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.112+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0de808b070 con 0x7f0df815db60 2023-11-28T11:11:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:19.310+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0df80742b0 con 0x7f0df815db60 2023-11-28T11:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.089+0000 7f0def7fe700 1 -- 172.21.15.83:0/2615127292 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0de8053860 con 0x7f0df815db60 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0dd80682f0 msgr2=0x7f0dd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0dd80682f0 0x7f0dd806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0df0003ef0 tx=0x7f0df00097f0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 msgr2=0x7f0df80c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 0x7f0df80c9f30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009130 tx=0x7f0de8004750 comp rx=0 tx=0).stop 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 shutdown_connections 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0dd80682f0 0x7f0dd806a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0df815db60 0x7f0df80c9f30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0df80b87e0 0x7f0df815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 --2- 172.21.15.83:0/2615127292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0df8160bc0 0x7f0df814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 >> 172.21.15.83:0/2615127292 conn(0x7f0df80b4ff0 msgr2=0x7f0df80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 shutdown_connections 2023-11-28T11:11:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.092+0000 7f0dfd193700 1 -- 172.21.15.83:0/2615127292 wait complete. 2023-11-28T11:11:20.094 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-28T11:11:20.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:11:20.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.230+0000 7fdaef68b700 1 Processor -- start 2023-11-28T11:11:20.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.231+0000 7fdaef68b700 1 -- start start 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.231+0000 7fdaef68b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdae815a8e0 0x7fdae80b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.231+0000 7fdaef68b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.231+0000 7fdaef68b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae815f1d0 con 0x7fdae815a510 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaef68b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae80cd780 con 0x7fdae815a8e0 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaef68b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80b3320 con 0x7fdae80b2ea0 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaecc26700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdae815a8e0 0x7fdae80b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaedc28700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaed427700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fdae815a510 0x7fdae80cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:48702/0 (socket says 172.21.15.83:48702) 2023-11-28T11:11:20.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaed427700 1 -- 172.21.15.83:0/3175560392 learned_addr learned my addr 172.21.15.83:0/3175560392 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaedc28700 1 -- 172.21.15.83:0/3175560392 >> v2:172.21.15.90:3300/0 conn(0x7fdae815a8e0 msgr2=0x7fdae80b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaedc28700 1 --2- 172.21.15.83:0/3175560392 >> v2:172.21.15.90:3300/0 conn(0x7fdae815a8e0 0x7fdae80b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaedc28700 1 -- 172.21.15.83:0/3175560392 >> v1:172.21.15.90:6789/0 conn(0x7fdae815a510 legacy=0x7fdae80cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaedc28700 1 -- 172.21.15.83:0/3175560392 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae815eef0 con 0x7fdae80b2ea0 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaecc26700 1 --2- 172.21.15.83:0/3175560392 >> v2:172.21.15.90:3300/0 conn(0x7fdae815a8e0 0x7fdae80b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.232+0000 7fdaedc28700 1 --2- 172.21.15.83:0/3175560392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae80b3d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009d90 tx=0x7fdae000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc2e6606c3e629 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdadffff700 1 -- 172.21.15.83:0/3175560392 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000e070 con 0x7fdae80b2ea0 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdadffff700 1 -- 172.21.15.83:0/3175560392 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae000bb20 con 0x7fdae80b2ea0 2023-11-28T11:11:20.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdadffff700 1 -- 172.21.15.83:0/3175560392 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0012430 con 0x7fdae80b2ea0 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 -- 172.21.15.83:0/3175560392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 msgr2=0x7fdae80b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3175560392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae80b3d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009d90 tx=0x7fdae000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 -- 172.21.15.83:0/3175560392 shutdown_connections 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3175560392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae80b3d30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3175560392 >> v2:172.21.15.90:3300/0 conn(0x7fdae815a8e0 0x7fdae80b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 -- 172.21.15.83:0/3175560392 >> 172.21.15.83:0/3175560392 conn(0x7fdae80cd960 msgr2=0x7fdae80cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 -- 172.21.15.83:0/3175560392 shutdown_connections 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.233+0000 7fdaef68b700 1 -- 172.21.15.83:0/3175560392 wait complete. 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.234+0000 7fdaef68b700 1 Processor -- start 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 -- start start 2023-11-28T11:11:20.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdae815a510 0x7fdae80b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 0x7fdae80b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8151c50 con 0x7fdae815a8e0 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae814f7b0 con 0x7fdae815a510 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaef68b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae814eba0 con 0x7fdae80b2ea0 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaedc28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 0x7fdae80b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaedc28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 0x7fdae80b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52506/0 (socket says 172.21.15.83:52506) 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaedc28700 1 -- 172.21.15.83:0/3893100488 learned_addr learned my addr 172.21.15.83:0/3893100488 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaecc26700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdae815a510 0x7fdae80b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.235+0000 7fdaed427700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaedc28700 1 -- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 msgr2=0x7fdae814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaedc28700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae814e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaedc28700 1 -- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdae815a510 msgr2=0x7fdae80b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaedc28700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdae815a510 0x7fdae80b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaedc28700 1 -- 172.21.15.83:0/3893100488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae40094a0 con 0x7fdae815a8e0 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaedc28700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 0x7fdae80b0e60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdae0014040 tx=0x7fdae0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0012430 con 0x7fdae815a8e0 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0020070 con 0x7fdae815a8e0 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0003ea0 con 0x7fdae815a8e0 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae0009a70 con 0x7fdae815a8e0 2023-11-28T11:11:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.236+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae80cf0b0 con 0x7fdae815a8e0 2023-11-28T11:11:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.237+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae80cedd0 con 0x7fdae815a8e0 2023-11-28T11:11:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.238+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdae002e070 con 0x7fdae815a8e0 2023-11-28T11:11:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.238+0000 7fdaddffb700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdacc0682f0 0x7fdacc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.238+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdae0022070 con 0x7fdae815a8e0 2023-11-28T11:11:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.238+0000 7fdaed427700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdacc0682f0 0x7fdacc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.239+0000 7fdaed427700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdacc0682f0 0x7fdacc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005910 tx=0x7fdad8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.242+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdae00547b0 con 0x7fdae815a8e0 2023-11-28T11:11:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.495+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdae80cf390 con 0x7fdae815a8e0 2023-11-28T11:11:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.495+0000 7fdaddffb700 1 -- 172.21.15.83:0/3893100488 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fdae002c070 con 0x7fdae815a8e0 2023-11-28T11:11:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVly5GSMRAArlawRsTlzQ8deoXsPnYmsg==", 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.497 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.498 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:11:20.498 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:11:20.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.498 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.499 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-28T11:11:20.499 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVl2bkZAhAA15xpS3O6vPGdnCqW98I/0w==", 2023-11-28T11:11:20.499 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.499 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.499 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:11:20.499 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlCEbQOBAA8FKGXdCurT68yNjSUEJigQ==", 2023-11-28T11:11:20.500 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.501 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.501 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:11:20.501 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:11:20.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlQFxUCRAA9BmlOX7cM5ewNextI+L6lg==", 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:11:20.502 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:11:20.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.503 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-28T11:11:20.503 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAAymVlMHf8JRAAqxuxtuaslP93VXhK17KfXA==", 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQADymVlUrMkJRAAxCUtzDNzfHRTJ4jLzeTa0Q==", 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.506 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-28T11:11:20.506 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAGymVlZqxOJBAAzhcB9ERpDVy5FCZNn1uMLg==", 2023-11-28T11:11:20.506 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.506 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-28T11:11:20.507 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAJymVlPZEhJBAAm8fhc9fRDDq3ZROV4Mez1Q==", 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.509 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-28T11:11:20.509 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAPymVlqwREKBAAsRlS/3ztcuWSZ9nRXZDKuA==", 2023-11-28T11:11:20.509 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.509 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.509 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQASymVlQKsoKBAABR2d0tZ/TLRKIDyQrAKVkA==", 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.510 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.511 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.511 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAZymVlqduLAhAAPau0X+UZigxTAoWpmUnzRg==", 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQAcymVlckzLBBAAH4vnm0MWED/F40ZWq2uIog==", 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVllJ2YDRAAE9HeK1zyWWnOIRh09fAG3Q==", 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:11:20.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.515 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-28T11:11:20.515 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVln8eKFhAAPGbMQ2n6PsaH6hEURGRJvw==", 2023-11-28T11:11:20.515 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.515 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlXJRCExAATH4e/FL5+nvYhdsZ7mB9LQ==", 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.516 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD9yWVlo7c6HBAAHpELpEY29uzkqcJ+jAaxRA==", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-28T11:11:20.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlC9hgFRAAL7ZVG6EzaROjM+to9qi5ZQ==", 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlBfMxLhAA6N4Cnno8jNhgzS5eYJKWqQ==", 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQD8yWVlHSGFKhAA3pr4QAikKKmqCfS0orTyWw==", 2023-11-28T11:11:20.519 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-28T11:11:20.520 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdacc0682f0 msgr2=0x7fdacc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdacc0682f0 0x7fdacc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005910 tx=0x7fdad8009040 comp rx=0 tx=0).stop 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 msgr2=0x7fdae80b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 0x7fdae80b0e60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdae0014040 tx=0x7fdae0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 shutdown_connections 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdacc0682f0 0x7fdacc06a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdae815a8e0 0x7fdae80b0e60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdae80b2ea0 0x7fdae814e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 --2- 172.21.15.83:0/3893100488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdae815a510 0x7fdae80b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 >> 172.21.15.83:0/3893100488 conn(0x7fdae80cd960 msgr2=0x7fdae81e40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.497+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 shutdown_connections 2023-11-28T11:11:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.498+0000 7fdaef68b700 1 -- 172.21.15.83:0/3893100488 wait complete. 2023-11-28T11:11:20.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.643+0000 7fa121478700 1 Processor -- start 2023-11-28T11:11:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.643+0000 7fa121478700 1 -- start start 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa121478700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa11c160690 0x7fa11c140240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa121478700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c06d500 0x7fa11c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa121478700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0c9e10 con 0x7fa11c15a560 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa121478700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c06de80 con 0x7fa11c160690 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa121478700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15d0a0 con 0x7fa11c06d500 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11a7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa11c160690 0x7fa11c140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa11c15a560 0x7fa11c142cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50370/0 (socket says 172.21.15.83:50370) 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11affd700 1 -- 172.21.15.83:0/2065751989 learned_addr learned my addr 172.21.15.83:0/2065751989 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11a7fc700 1 -- 172.21.15.83:0/2065751989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c06d500 msgr2=0x7fa11c06d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11b7fe700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c06d500 0x7fa11c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa119ffb700 1 -- 172.21.15.83:0/2065751989 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760759875 0 0) 0x7fa11c0c9e10 con 0x7fa11c15a560 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11a7fc700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c06d500 0x7fa11c06d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11a7fc700 1 -- 172.21.15.83:0/2065751989 >> v1:172.21.15.83:6789/0 conn(0x7fa11c15a560 legacy=0x7fa11c142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11a7fc700 1 -- 172.21.15.83:0/2065751989 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c160b90 con 0x7fa11c160690 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11b7fe700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c06d500 0x7fa11c06d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.644+0000 7fa11a7fc700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c160690 0x7fa11c140240 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009840 tx=0x7fa10c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78775ba1e69a355 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa119ffb700 1 -- 172.21.15.83:0/2065751989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c011830 con 0x7fa11c160690 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa119ffb700 1 -- 172.21.15.83:0/2065751989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c011990 con 0x7fa11c160690 2023-11-28T11:11:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa119ffb700 1 -- 172.21.15.83:0/2065751989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c011d20 con 0x7fa11c160690 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 -- 172.21.15.83:0/2065751989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c160690 msgr2=0x7fa11c140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c160690 0x7fa11c140240 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009840 tx=0x7fa10c009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 -- 172.21.15.83:0/2065751989 shutdown_connections 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c06d500 0x7fa11c06d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 --2- 172.21.15.83:0/2065751989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c160690 0x7fa11c140240 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 -- 172.21.15.83:0/2065751989 >> 172.21.15.83:0/2065751989 conn(0x7fa11c0ccb40 msgr2=0x7fa11c0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 -- 172.21.15.83:0/2065751989 shutdown_connections 2023-11-28T11:11:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.645+0000 7fa121478700 1 -- 172.21.15.83:0/2065751989 wait complete. 2023-11-28T11:11:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 Processor -- start 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 -- start start 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c06d500 0x7fa11c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 0x7fa11c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c160690 0x7fa11c1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0b16d0 con 0x7fa11c15a560 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0b22a0 con 0x7fa11c06d500 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa121478700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0b2580 con 0x7fa11c160690 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa11a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 0x7fa11c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa11a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 0x7fa11c155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52518/0 (socket says 172.21.15.83:52518) 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.647+0000 7fa11a7fc700 1 -- 172.21.15.83:0/860081265 learned_addr learned my addr 172.21.15.83:0/860081265 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11b7fe700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c160690 0x7fa11c1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11affd700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c06d500 0x7fa11c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11a7fc700 1 -- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c160690 msgr2=0x7fa11c1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11a7fc700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c160690 0x7fa11c1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11a7fc700 1 -- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c06d500 msgr2=0x7fa11c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11a7fc700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c06d500 0x7fa11c0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11a7fc700 1 -- 172.21.15.83:0/860081265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0094a0 con 0x7fa11c15a560 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa11a7fc700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 0x7fa11c155ca0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0090b0 tx=0x7fa10c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0038a0 con 0x7fa11c15a560 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c01f070 con 0x7fa11c15a560 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c161150 con 0x7fa11c15a560 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c003c40 con 0x7fa11c15a560 2023-11-28T11:11:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.648+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c160e70 con 0x7fa11c15a560 2023-11-28T11:11:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.649+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c161430 con 0x7fa11c15a560 2023-11-28T11:11:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.649+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa10c00f050 con 0x7fa11c15a560 2023-11-28T11:11:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.650+0000 7fa1037fe700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1080682f0 0x7fa10806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.650+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa10c021070 con 0x7fa11c15a560 2023-11-28T11:11:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.650+0000 7fa11affd700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1080682f0 0x7fa10806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.650+0000 7fa11affd700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1080682f0 0x7fa10806a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa110000f60 tx=0x7fa110007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.654+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa10c053fb0 con 0x7fa11c15a560 2023-11-28T11:11:20.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.838+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa11c0b2aa0 con 0x7fa11c15a560 2023-11-28T11:11:20.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.839+0000 7fa1037fe700 1 -- 172.21.15.83:0/860081265 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+14366 (secure 0 0 0) 0x7fa10c084830 con 0x7fa11c15a560 2023-11-28T11:11:20.839 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:20.840 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:11:20.083240+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:20.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1080682f0 msgr2=0x7fa10806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1080682f0 0x7fa10806a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa110000f60 tx=0x7fa110007be0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 msgr2=0x7fa11c155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 0x7fa11c155ca0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0090b0 tx=0x7fa10c004750 comp rx=0 tx=0).stop 2023-11-28T11:11:20.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 shutdown_connections 2023-11-28T11:11:20.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa11c160690 0x7fa11c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c06d500 0x7fa11c0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa1080682f0 0x7fa10806a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 --2- 172.21.15.83:0/860081265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa11c15a560 0x7fa11c155ca0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 >> 172.21.15.83:0/860081265 conn(0x7fa11c0ccb40 msgr2=0x7fa11c159770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 shutdown_connections 2023-11-28T11:11:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.841+0000 7fa121478700 1 -- 172.21.15.83:0/860081265 wait complete. 2023-11-28T11:11:20.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:11:20.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:11:20.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.976+0000 7fe41f2bc700 1 Processor -- start 2023-11-28T11:11:20.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 -- start start 2023-11-28T11:11:20.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41815c890 0x7fe4180aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4180b4bb0 0x7fe41806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe41806e1a0 0x7fe4181604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41806e620 con 0x7fe4180b4bb0 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4180b5020 con 0x7fe41806e1a0 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.977+0000 7fe41f2bc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41806d4a0 con 0x7fe41815c890 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4180b4bb0 0x7fe41806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4180b4bb0 0x7fe41806dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52524/0 (socket says 172.21.15.83:52524) 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 -- 172.21.15.83:0/2113815532 learned_addr learned my addr 172.21.15.83:0/2113815532 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41d859700 1 --2- 172.21.15.83:0/2113815532 >> v2:172.21.15.90:3300/0 conn(0x7fe41806e1a0 0x7fe4181604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41d058700 1 --2- 172.21.15.83:0/2113815532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41815c890 0x7fe4180aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 -- 172.21.15.83:0/2113815532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41815c890 msgr2=0x7fe4180aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 --2- 172.21.15.83:0/2113815532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41815c890 0x7fe4180aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 -- 172.21.15.83:0/2113815532 >> v2:172.21.15.90:3300/0 conn(0x7fe41806e1a0 msgr2=0x7fe4181604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 --2- 172.21.15.83:0/2113815532 >> v2:172.21.15.90:3300/0 conn(0x7fe41806e1a0 0x7fe4181604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 -- 172.21.15.83:0/2113815532 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180b5ae0 con 0x7fe4180b4bb0 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe41c857700 1 --2- 172.21.15.83:0/2113815532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4180b4bb0 0x7fe41806dc60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098f0 tx=0x7fe414009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c80b9f55c8c035 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe40ffff700 1 -- 172.21.15.83:0/2113815532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41400b860 con 0x7fe4180b4bb0 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe40ffff700 1 -- 172.21.15.83:0/2113815532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41400b9c0 con 0x7fe4180b4bb0 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.978+0000 7fe40ffff700 1 -- 172.21.15.83:0/2113815532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41400bd50 con 0x7fe4180b4bb0 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 -- 172.21.15.83:0/2113815532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4180b4bb0 msgr2=0x7fe41806dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/2113815532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4180b4bb0 0x7fe41806dc60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098f0 tx=0x7fe414009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 -- 172.21.15.83:0/2113815532 shutdown_connections 2023-11-28T11:11:20.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/2113815532 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41815c890 0x7fe4180aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/2113815532 >> v2:172.21.15.90:3300/0 conn(0x7fe41806e1a0 0x7fe4181604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/2113815532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4180b4bb0 0x7fe41806dc60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 -- 172.21.15.83:0/2113815532 >> 172.21.15.83:0/2113815532 conn(0x7fe4180b2b60 msgr2=0x7fe4180b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:20.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 -- 172.21.15.83:0/2113815532 shutdown_connections 2023-11-28T11:11:20.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.979+0000 7fe41f2bc700 1 -- 172.21.15.83:0/2113815532 wait complete. 2023-11-28T11:11:20.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.980+0000 7fe41f2bc700 1 Processor -- start 2023-11-28T11:11:20.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.980+0000 7fe41f2bc700 1 -- start start 2023-11-28T11:11:20.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41f2bc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41806e1a0 0x7fe4181f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41f2bc700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180b4bb0 0x7fe418155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41f2bc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 0x7fe4180ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41f2bc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180ca030 con 0x7fe41815c890 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41f2bc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180c8d90 con 0x7fe41806e1a0 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41f2bc700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181f30b0 con 0x7fe4180b4bb0 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41d859700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 0x7fe4180ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41d859700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 0x7fe4180ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52534/0 (socket says 172.21.15.83:52534) 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.981+0000 7fe41d859700 1 -- 172.21.15.83:0/1302480889 learned_addr learned my addr 172.21.15.83:0/1302480889 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d058700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41806e1a0 0x7fe4181f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41c857700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180b4bb0 0x7fe418155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d859700 1 -- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180b4bb0 msgr2=0x7fe418155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d859700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180b4bb0 0x7fe418155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d859700 1 -- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41806e1a0 msgr2=0x7fe4181f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d859700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41806e1a0 0x7fe4181f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d859700 1 -- 172.21.15.83:0/1302480889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414009550 con 0x7fe41815c890 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d058700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41806e1a0 0x7fe4181f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41d859700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 0x7fe4180ce6a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe410007d50 tx=0x7fe41000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000ede0 con 0x7fe41815c890 2023-11-28T11:11:20.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410014420 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4180b5240 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100147d0 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.982+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41806c400 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.983+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4180b5520 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.983+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe410012050 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.984+0000 7fe40dffb700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3fc0682f0 0x7fe3fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.984+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe4100860c0 con 0x7fe41815c890 2023-11-28T11:11:20.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.984+0000 7fe41d058700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3fc0682f0 0x7fe3fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:20.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.984+0000 7fe41d058700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3fc0682f0 0x7fe3fc06a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe408006fd0 tx=0x7fe408008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:20.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:20.988+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe410052a40 con 0x7fe41815c890 2023-11-28T11:11:21.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.171+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe418071790 con 0x7fe41815c890 2023-11-28T11:11:21.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.171+0000 7fe40dffb700 1 -- 172.21.15.83:0/1302480889 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe410056060 con 0x7fe41815c890 2023-11-28T11:11:21.172 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3fc0682f0 msgr2=0x7fe3fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3fc0682f0 0x7fe3fc06a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe408006fd0 tx=0x7fe408008040 comp rx=0 tx=0).stop 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 msgr2=0x7fe4180ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 0x7fe4180ce6a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe410007d50 tx=0x7fe41000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 shutdown_connections 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe3fc0682f0 0x7fe3fc06a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe41815c890 0x7fe4180ce6a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41806e1a0 0x7fe4181f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 --2- 172.21.15.83:0/1302480889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180b4bb0 0x7fe418155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 >> 172.21.15.83:0/1302480889 conn(0x7fe4180b2b60 msgr2=0x7fe41815b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:21.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 shutdown_connections 2023-11-28T11:11:21.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.173+0000 7fe41f2bc700 1 -- 172.21.15.83:0/1302480889 wait complete. 2023-11-28T11:11:21.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:11:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.308+0000 7f29c6ffa700 1 Processor -- start 2023-11-28T11:11:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.308+0000 7f29c6ffa700 1 -- start start 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c6ffa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 0x7f29c0160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c6ffa700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f29c015d4d0 0x7f29c00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c6ffa700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c0157120 con 0x7f29c00cea20 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c6ffa700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00cbc70 con 0x7f29c015d4d0 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c6ffa700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c015d040 con 0x7f29c00cc020 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 0x7f29c0160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c5597700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f29c015d4d0 0x7f29c00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 0x7f29c0160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:36740/0 (socket says 172.21.15.83:36740) 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 -- 172.21.15.83:0/561839796 learned_addr learned my addr 172.21.15.83:0/561839796 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 -- 172.21.15.83:0/561839796 >> v2:172.21.15.90:3300/0 conn(0x7f29c015d4d0 msgr2=0x7f29c00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 --2- 172.21.15.83:0/561839796 >> v2:172.21.15.90:3300/0 conn(0x7f29c015d4d0 0x7f29c00ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 -- 172.21.15.83:0/561839796 >> v1:172.21.15.90:6789/0 conn(0x7f29c00cea20 legacy=0x7f29c00cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.309+0000 7f29c4d96700 1 -- 172.21.15.83:0/561839796 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00ca3c0 con 0x7f29c00cc020 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c4d96700 1 --2- 172.21.15.83:0/561839796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 0x7f29c0160270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b40098f0 tx=0x7f29b4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d486bea26283acf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29bf7fe700 1 -- 172.21.15.83:0/561839796 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4011830 con 0x7f29c00cc020 2023-11-28T11:11:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29bf7fe700 1 -- 172.21.15.83:0/561839796 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4011990 con 0x7f29c00cc020 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29bf7fe700 1 -- 172.21.15.83:0/561839796 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4011d20 con 0x7f29c00cc020 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 -- 172.21.15.83:0/561839796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 msgr2=0x7f29c0160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/561839796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 0x7f29c0160270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b40098f0 tx=0x7f29b4009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 -- 172.21.15.83:0/561839796 shutdown_connections 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/561839796 >> v2:172.21.15.90:3300/0 conn(0x7f29c015d4d0 0x7f29c00ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/561839796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cc020 0x7f29c0160270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 -- 172.21.15.83:0/561839796 >> 172.21.15.83:0/561839796 conn(0x7f29c00b1090 msgr2=0x7f29c00b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 -- 172.21.15.83:0/561839796 shutdown_connections 2023-11-28T11:11:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.310+0000 7f29c6ffa700 1 -- 172.21.15.83:0/561839796 wait complete. 2023-11-28T11:11:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 Processor -- start 2023-11-28T11:11:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 -- start start 2023-11-28T11:11:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 0x7f29c00c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cea20 0x7f29c0151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c015d4d0 0x7f29c00c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c5b60 con 0x7f29c00cc020 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c5d70 con 0x7f29c015d4d0 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.312+0000 7f29c6ffa700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c3990 con 0x7f29c00cea20 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 0x7f29c00c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 0x7f29c00c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52544/0 (socket says 172.21.15.83:52544) 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 -- 172.21.15.83:0/1126130553 learned_addr learned my addr 172.21.15.83:0/1126130553 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c5597700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c015d4d0 0x7f29c00c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29bffff700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cea20 0x7f29c0151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 -- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cea20 msgr2=0x7f29c0151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cea20 0x7f29c0151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 -- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c015d4d0 msgr2=0x7f29c00c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c015d4d0 0x7f29c00c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 -- 172.21.15.83:0/1126130553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b4009550 con 0x7f29c00cc020 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c4d96700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 0x7f29c00c5620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000c00 tx=0x7f29b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4004570 con 0x7f29c00cc020 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b401f070 con 0x7f29c00cc020 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c0160a50 con 0x7f29c00cc020 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40038a0 con 0x7f29c00cc020 2023-11-28T11:11:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.313+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c01607b0 con 0x7f29c00cc020 2023-11-28T11:11:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.314+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c0160d30 con 0x7f29c00cc020 2023-11-28T11:11:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.315+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f29b402a020 con 0x7f29c00cc020 2023-11-28T11:11:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.315+0000 7f29bd7fa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29a40682f0 0x7f29a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.315+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f29b4087060 con 0x7f29c00cc020 2023-11-28T11:11:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.315+0000 7f29bffff700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29a40682f0 0x7f29a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.316+0000 7f29bffff700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29a40682f0 0x7f29a406a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f29c015f840 tx=0x7f29ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.319+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f29b4053ad0 con 0x7f29c00cc020 2023-11-28T11:11:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.500+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29c00742b0 con 0x7f29c00cc020 2023-11-28T11:11:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.501+0000 7f29bd7fa700 1 -- 172.21.15.83:0/1126130553 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29b40150a0 con 0x7f29c00cc020 2023-11-28T11:11:21.501 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:11:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29a40682f0 msgr2=0x7f29a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29a40682f0 0x7f29a406a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f29c015f840 tx=0x7f29ac009040 comp rx=0 tx=0).stop 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 msgr2=0x7f29c00c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 0x7f29c00c5620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000c00 tx=0x7f29b4004750 comp rx=0 tx=0).stop 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 shutdown_connections 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c015d4d0 0x7f29c00c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29a40682f0 0x7f29a406a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c00cc020 0x7f29c00c5620 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 --2- 172.21.15.83:0/1126130553 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29c00cea20 0x7f29c0151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 >> 172.21.15.83:0/1126130553 conn(0x7f29c00b1090 msgr2=0x7f29c006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 shutdown_connections 2023-11-28T11:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.503+0000 7f29c6ffa700 1 -- 172.21.15.83:0/1126130553 wait complete. 2023-11-28T11:11:21.516 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.636+0000 7fc6dd1aa700 1 Processor -- start 2023-11-28T11:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.636+0000 7fc6dd1aa700 1 -- start start 2023-11-28T11:11:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6dd1aa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 0x7fc6d8160e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6dd1aa700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d80b0790 con 0x7fc6d815cab0 2023-11-28T11:11:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6dd1aa700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d806fbe0 con 0x7fc6d806d3e0 2023-11-28T11:11:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6dd1aa700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d806fec0 con 0x7fc6d815c6e0 2023-11-28T11:11:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc6d815cab0 0x7fc6d80cb530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50412/0 (socket says 172.21.15.83:50412) 2023-11-28T11:11:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d659c700 1 -- 172.21.15.83:0/2920081964 learned_addr learned my addr 172.21.15.83:0/2920081964 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724439031 0 0) 0x7fc6d80b0790 con 0x7fc6d815cab0 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d6d9d700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 0x7fc6d8160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c00035b0 con 0x7fc6d815cab0 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668425858 0 0) 0x7fc6d806fbe0 con 0x7fc6d806d3e0 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d80b0790 con 0x7fc6d806d3e0 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3565987386 0 0) 0x7fc6c00035b0 con 0x7fc6d815cab0 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 msgr2=0x7fc6d8160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 0x7fc6d8160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 >> v1:172.21.15.90:6789/0 conn(0x7fc6d806d3e0 legacy=0x7fc6d806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d806bec0 con 0x7fc6d815cab0 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fc6cc001670 con 0x7fc6d815cab0 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 >> v1:172.21.15.83:6789/0 conn(0x7fc6d815cab0 legacy=0x7fc6d80cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 0x7fc6c0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c0004170 0x7fc6c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c0004d90 0x7fc6c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d806fec0 con 0x7fc6c0004d90 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c000bb20 con 0x7fc6c0004170 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.637+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c000bce0 con 0x7fc6d815c6e0 2023-11-28T11:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c0004d90 0x7fc6c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d6d9d700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 0x7fc6c0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d659c700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c0004170 0x7fc6c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 -- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 msgr2=0x7fc6c0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d815c6e0 0x7fc6c0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 -- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c0004170 msgr2=0x7fc6c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c0004170 0x7fc6c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 -- 172.21.15.83:0/2920081964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00035b0 con 0x7fc6c0004d90 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d759e700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c0004d90 0x7fc6c0005200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c2f0 tx=0x7fc6c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c667950a5d80bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0001f30 con 0x7fc6c0004d90 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d000ce90 con 0x7fc6c0004d90 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.638+0000 7fc6d5d9b700 1 -- 172.21.15.83:0/2920081964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0003c40 con 0x7fc6c0004d90 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/2920081964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c0004d90 msgr2=0x7fc6c0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/2920081964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c0004d90 0x7fc6c0005200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c2f0 tx=0x7fc6c0005880 comp rx=0 tx=0).stop 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6d759e700 1 -- 172.21.15.83:0/2920081964 reap_dead start 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/2920081964 shutdown_connections 2023-11-28T11:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/2920081964 >> 172.21.15.83:0/2920081964 conn(0x7fc6d806de20 msgr2=0x7fc6d806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/2920081964 shutdown_connections 2023-11-28T11:11:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.639+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/2920081964 wait complete. 2023-11-28T11:11:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.640+0000 7fc6dd1aa700 1 Processor -- start 2023-11-28T11:11:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.640+0000 7fc6dd1aa700 1 -- start start 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6dd1aa700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d80c8cd0 0x7fc6d80c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6dd1aa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 0x7fc6d815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6dd1aa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6d815f5e0 0x7fc6d80c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6dd1aa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d815ba50 con 0x7fc6d80c4690 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6dd1aa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d815e930 con 0x7fc6d815f5e0 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6dd1aa700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80c4b00 con 0x7fc6d80c8cd0 2023-11-28T11:11:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 0x7fc6d815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 0x7fc6d815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52566/0 (socket says 172.21.15.83:52566) 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 -- 172.21.15.83:0/3683457323 learned_addr learned my addr 172.21.15.83:0/3683457323 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d6d9d700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d80c8cd0 0x7fc6d80c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d759e700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6d815f5e0 0x7fc6d80c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 -- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d80c8cd0 msgr2=0x7fc6d80c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d80c8cd0 0x7fc6d80c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 -- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6d815f5e0 msgr2=0x7fc6d80c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6d815f5e0 0x7fc6d80c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 -- 172.21.15.83:0/3683457323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80cbc40 con 0x7fc6d80c4690 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.641+0000 7fc6d659c700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 0x7fc6d815f0a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc007290 tx=0x7fc6cc007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.642+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00f6c0 con 0x7fc6d80c4690 2023-11-28T11:11:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.642+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc00f820 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.642+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d806c1a0 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.642+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00fb90 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.642+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d80701a0 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.643+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d8074320 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.643+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6cc00fcf0 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.643+0000 7fc6bf7fe700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc6c40682f0 0x7fc6c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.644+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc6cc085c50 con 0x7fc6d80c4690 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.644+0000 7fc6d6d9d700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc6c40682f0 0x7fc6c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.644+0000 7fc6d6d9d700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc6c40682f0 0x7fc6c406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80c7ee0 tx=0x7fc6c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.648+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6cc0526c0 con 0x7fc6d80c4690 2023-11-28T11:11:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.832+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc6d80b6e60 con 0x7fc6d80c4690 2023-11-28T11:11:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.832+0000 7fc6bf7fe700 1 -- 172.21.15.83:0/3683457323 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc6cc055ce0 con 0x7fc6d80c4690 2023-11-28T11:11:21.833 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:21.833 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-28T11:11:21.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:21.834 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-28T11:11:21.834 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": "rbd" 2023-11-28T11:11:21.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:21.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:21.835 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 2, 2023-11-28T11:11:21.835 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-28T11:11:21.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:21.835 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-28T11:11:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc6c40682f0 msgr2=0x7fc6c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc6c40682f0 0x7fc6c406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80c7ee0 tx=0x7fc6c8008040 comp rx=0 tx=0).stop 2023-11-28T11:11:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 msgr2=0x7fc6d815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 0x7fc6d815f0a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc007290 tx=0x7fc6cc007650 comp rx=0 tx=0).stop 2023-11-28T11:11:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 shutdown_connections 2023-11-28T11:11:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6d815f5e0 0x7fc6d80c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc6d80c8cd0 0x7fc6d80c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc6c40682f0 0x7fc6c406a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.834+0000 7fc6dd1aa700 1 --2- 172.21.15.83:0/3683457323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d80c4690 0x7fc6d815f0a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.835+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 >> 172.21.15.83:0/3683457323 conn(0x7fc6d806de20 msgr2=0x7fc6d80b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:21.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.835+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 shutdown_connections 2023-11-28T11:11:21.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.835+0000 7fc6dd1aa700 1 -- 172.21.15.83:0/3683457323 wait complete. 2023-11-28T11:11:21.848 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:11:21.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:11:21.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.971+0000 7f7604af9700 1 Processor -- start 2023-11-28T11:11:21.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.971+0000 7f7604af9700 1 -- start start 2023-11-28T11:11:21.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f7604af9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76000ccfd0 0x7f76001675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f7604af9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000cbb30 con 0x7f76000cf040 2023-11-28T11:11:21.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f7604af9700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76001611f0 con 0x7f760015c6d0 2023-11-28T11:11:21.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f7604af9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000701b0 con 0x7f76000ccfd0 2023-11-28T11:11:21.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f75fed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76000ccfd0 0x7f76001675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f75fdd9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f76000cf040 0x7f76001602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50434/0 (socket says 172.21.15.83:50434) 2023-11-28T11:11:21.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f75fdd9b700 1 -- 172.21.15.83:0/170787221 learned_addr learned my addr 172.21.15.83:0/170787221 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:21.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f75fed9d700 1 -- 172.21.15.83:0/170787221 >> v1:172.21.15.90:6789/0 conn(0x7f760015c6d0 legacy=0x7f76000ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f75fed9d700 1 -- 172.21.15.83:0/170787221 >> v1:172.21.15.83:6789/0 conn(0x7f76000cf040 legacy=0x7f76001602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.972+0000 7f75fed9d700 1 -- 172.21.15.83:0/170787221 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600160f10 con 0x7f76000ccfd0 2023-11-28T11:11:21.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.973+0000 7f75fed9d700 1 --2- 172.21.15.83:0/170787221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f76001675f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009d90 tx=0x7f75f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa0c6db8a3c821a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.976+0000 7f75fd59a700 1 -- 172.21.15.83:0/170787221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800e070 con 0x7f76000ccfd0 2023-11-28T11:11:21.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.976+0000 7f75fd59a700 1 -- 172.21.15.83:0/170787221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f800bbd0 con 0x7f76000ccfd0 2023-11-28T11:11:21.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.976+0000 7f75fd59a700 1 -- 172.21.15.83:0/170787221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8012430 con 0x7f76000ccfd0 2023-11-28T11:11:21.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 -- 172.21.15.83:0/170787221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 msgr2=0x7f76001675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 --2- 172.21.15.83:0/170787221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f76001675f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009d90 tx=0x7f75f800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:21.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 -- 172.21.15.83:0/170787221 shutdown_connections 2023-11-28T11:11:21.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 --2- 172.21.15.83:0/170787221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f76001675f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 -- 172.21.15.83:0/170787221 >> 172.21.15.83:0/170787221 conn(0x7f760006de20 msgr2=0x7f760006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:21.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 -- 172.21.15.83:0/170787221 shutdown_connections 2023-11-28T11:11:21.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.977+0000 7f7604af9700 1 -- 172.21.15.83:0/170787221 wait complete. 2023-11-28T11:11:21.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.978+0000 7f7604af9700 1 Processor -- start 2023-11-28T11:11:21.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.978+0000 7f7604af9700 1 -- start start 2023-11-28T11:11:21.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f7604af9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f760015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f7604af9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76000cf040 0x7f760014d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f7604af9700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f760015c6d0 0x7f76000c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f7604af9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000cabb0 con 0x7f76000ccfd0 2023-11-28T11:11:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f7604af9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000c4890 con 0x7f76000cf040 2023-11-28T11:11:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f7604af9700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000c4b70 con 0x7f760015c6d0 2023-11-28T11:11:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f75fe59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f760015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f75fe59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f760015b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52578/0 (socket says 172.21.15.83:52578) 2023-11-28T11:11:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f75fe59c700 1 -- 172.21.15.83:0/3424046978 learned_addr learned my addr 172.21.15.83:0/3424046978 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f75fed9d700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f760015c6d0 0x7f76000c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.979+0000 7f75fdd9b700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76000cf040 0x7f760014d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.980+0000 7f75fe59c700 1 -- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f760015c6d0 msgr2=0x7f76000c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.980+0000 7f75fe59c700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f760015c6d0 0x7f76000c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.980+0000 7f75fe59c700 1 -- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76000cf040 msgr2=0x7f760014d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.980+0000 7f75fe59c700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76000cf040 0x7f760014d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.980+0000 7f75fe59c700 1 -- 172.21.15.83:0/3424046978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8009a70 con 0x7f76000ccfd0 2023-11-28T11:11:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.980+0000 7f75fe59c700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f760015b8d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c4e0 tx=0x7f75f000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.981+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00052d0 con 0x7f76000ccfd0 2023-11-28T11:11:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.981+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0005430 con 0x7f76000ccfd0 2023-11-28T11:11:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.981+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76000cb7c0 con 0x7f76000ccfd0 2023-11-28T11:11:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.981+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00038c0 con 0x7f76000ccfd0 2023-11-28T11:11:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.981+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000cb4e0 con 0x7f76000ccfd0 2023-11-28T11:11:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.982+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760015caa0 con 0x7f76000ccfd0 2023-11-28T11:11:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.982+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75f0019030 con 0x7f76000ccfd0 2023-11-28T11:11:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.982+0000 7f75eeffd700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75e00682f0 0x7f75e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.983+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f75f0084cd0 con 0x7f76000ccfd0 2023-11-28T11:11:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.983+0000 7f75fdd9b700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75e00682f0 0x7f75e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.983+0000 7f75fdd9b700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75e00682f0 0x7f75e006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75f4005910 tx=0x7f75f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:21.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:21.990+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75f0051740 con 0x7f76000ccfd0 2023-11-28T11:11:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.176+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76000742b0 con 0x7f76000ccfd0 2023-11-28T11:11:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.988+0000 7f75eeffd700 1 -- 172.21.15.83:0/3424046978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f75f00170b0 con 0x7f76000ccfd0 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.992+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75e00682f0 msgr2=0x7f75e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.992+0000 7f7604af9700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75e00682f0 0x7f75e006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75f4005910 tx=0x7f75f4009040 comp rx=0 tx=0).stop 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.992+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 msgr2=0x7f760015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.992+0000 7f7604af9700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f760015b8d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c4e0 tx=0x7f75f000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 shutdown_connections 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f760015c6d0 0x7f76000c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75e00682f0 0x7f75e006a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76000ccfd0 0x7f760015b8d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 --2- 172.21.15.83:0/3424046978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76000cf040 0x7f760014d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 >> 172.21.15.83:0/3424046978 conn(0x7f760006de20 msgr2=0x7f76000b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 shutdown_connections 2023-11-28T11:11:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:22.993+0000 7f7604af9700 1 -- 172.21.15.83:0/3424046978 wait complete. 2023-11-28T11:11:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-28T11:11:23.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:11:23.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.143+0000 7f76446e6700 1 Processor -- start 2023-11-28T11:11:23.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.143+0000 7f76446e6700 1 -- start start 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f76446e6700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f763c0cbb20 0x7f763c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f76446e6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f763c161230 0x7f763c160620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f76446e6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c0cd380 con 0x7f763c0cbef0 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f76446e6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0cc770 con 0x7f763c161230 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f76446e6700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0cca50 con 0x7f763c0cbb20 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f763c161230 0x7f763c160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7642c83700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f763c0cbef0 0x7f763c0aea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50450/0 (socket says 172.21.15.83:50450) 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f763c161230 0x7f763c160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52588/0 (socket says 172.21.15.83:52588) 2023-11-28T11:11:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7642c83700 1 -- 172.21.15.83:0/461517557 learned_addr learned my addr 172.21.15.83:0/461517557 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7642482700 1 --2- 172.21.15.83:0/461517557 >> v2:172.21.15.90:3300/0 conn(0x7f763c0cbb20 0x7f763c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 -- 172.21.15.83:0/461517557 >> v2:172.21.15.90:3300/0 conn(0x7f763c0cbb20 msgr2=0x7f763c0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 --2- 172.21.15.83:0/461517557 >> v2:172.21.15.90:3300/0 conn(0x7f763c0cbb20 0x7f763c0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 -- 172.21.15.83:0/461517557 >> v1:172.21.15.83:6789/0 conn(0x7f763c0cbef0 legacy=0x7f763c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 -- 172.21.15.83:0/461517557 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0ccd30 con 0x7f763c161230 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641480700 1 -- 172.21.15.83:0/461517557 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445612606 0 0) 0x7f763c0cd380 con 0x7f763c0cbef0 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.144+0000 7f7641c81700 1 --2- 172.21.15.83:0/461517557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c161230 0x7f763c160620 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f763c0b0e20 tx=0x7f762c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ebbd15b9e95d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f7641480700 1 -- 172.21.15.83:0/461517557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c011830 con 0x7f763c161230 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f7641480700 1 -- 172.21.15.83:0/461517557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c011990 con 0x7f763c161230 2023-11-28T11:11:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f7641480700 1 -- 172.21.15.83:0/461517557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c011d20 con 0x7f763c161230 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 -- 172.21.15.83:0/461517557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c161230 msgr2=0x7f763c160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 --2- 172.21.15.83:0/461517557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c161230 0x7f763c160620 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f763c0b0e20 tx=0x7f762c009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 -- 172.21.15.83:0/461517557 shutdown_connections 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 --2- 172.21.15.83:0/461517557 >> v2:172.21.15.90:3300/0 conn(0x7f763c0cbb20 0x7f763c0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 --2- 172.21.15.83:0/461517557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c161230 0x7f763c160620 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 -- 172.21.15.83:0/461517557 >> 172.21.15.83:0/461517557 conn(0x7f763c0b2ca0 msgr2=0x7f763c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 -- 172.21.15.83:0/461517557 shutdown_connections 2023-11-28T11:11:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.145+0000 7f76446e6700 1 -- 172.21.15.83:0/461517557 wait complete. 2023-11-28T11:11:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.146+0000 7f76446e6700 1 Processor -- start 2023-11-28T11:11:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.146+0000 7f76446e6700 1 -- start start 2023-11-28T11:11:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f76446e6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f763c0cbb20 0x7f763c1f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f76446e6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 0x7f763c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f76446e6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f763c161230 0x7f763c1f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f76446e6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c143960 con 0x7f763c0cbef0 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f76446e6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1f33c0 con 0x7f763c0cbb20 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f76446e6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1f2e00 con 0x7f763c161230 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f7641c81700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 0x7f763c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f7641c81700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 0x7f763c06c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52594/0 (socket says 172.21.15.83:52594) 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f7641c81700 1 -- 172.21.15.83:0/1668970828 learned_addr learned my addr 172.21.15.83:0/1668970828 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f7642c83700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f763c161230 0x7f763c1f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.147+0000 7f7642482700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f763c0cbb20 0x7f763c1f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7641c81700 1 -- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f763c161230 msgr2=0x7f763c1f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7641c81700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f763c161230 0x7f763c1f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7641c81700 1 -- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f763c0cbb20 msgr2=0x7f763c1f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7641c81700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f763c0cbb20 0x7f763c1f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7641c81700 1 -- 172.21.15.83:0/1668970828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c0094a0 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7642482700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f763c0cbb20 0x7f763c1f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7641c81700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 0x7f763c06c480 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f763c072910 tx=0x7f762c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c004590 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c0036a0 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c0b26e0 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c003a10 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.148+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c0cd010 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.149+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c160b60 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.149+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f762c00f050 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.150+0000 7f7632ffd700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76200682f0 0x7f762006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.150+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f762c086ee0 con 0x7f763c0cbef0 2023-11-28T11:11:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.150+0000 7f7642482700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76200682f0 0x7f762006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:23.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.150+0000 7f7642482700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76200682f0 0x7f762006a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7638009b90 tx=0x7f7638008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:23.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.154+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f762c053760 con 0x7f763c0cbef0 2023-11-28T11:11:23.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.336+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f763c074240 con 0x7f763c0cbef0 2023-11-28T11:11:23.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.990+0000 7f7632ffd700 1 -- 172.21.15.83:0/1668970828 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7f762c056d80 con 0x7f763c0cbef0 2023-11-28T11:11:23.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.998+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76200682f0 msgr2=0x7f762006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:23.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.998+0000 7f76446e6700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76200682f0 0x7f762006a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7638009b90 tx=0x7f7638008040 comp rx=0 tx=0).stop 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.998+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 msgr2=0x7f763c06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.998+0000 7f76446e6700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 0x7f763c06c480 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f763c072910 tx=0x7f762c004750 comp rx=0 tx=0).stop 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 shutdown_connections 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f763c161230 0x7f763c1f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f763c0cbb20 0x7f763c1f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76200682f0 0x7f762006a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 --2- 172.21.15.83:0/1668970828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c0cbef0 0x7f763c06c480 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 >> 172.21.15.83:0/1668970828 conn(0x7f763c0b2ca0 msgr2=0x7f763c0cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 shutdown_connections 2023-11-28T11:11:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:23.999+0000 7f76446e6700 1 -- 172.21.15.83:0/1668970828 wait complete. 2023-11-28T11:11:24.003 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-28T11:11:24.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:11:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.142+0000 7f6e919a0700 1 Processor -- start 2023-11-28T11:11:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.142+0000 7f6e919a0700 1 -- start start 2023-11-28T11:11:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e919a0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b8570 0x7f6e8c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e919a0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6e8c0d2880 0x7f6e8c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e919a0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e8c0b4940 0x7f6e8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e919a0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c0b52d0 con 0x7f6e8c0b4940 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e919a0700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c0cd520 con 0x7f6e8c0d2880 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e919a0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0cf400 con 0x7f6e8c0b8570 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e8c0b4940 0x7f6e8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e8c0b4940 0x7f6e8c0b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52602/0 (socket says 172.21.15.83:52602) 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 -- 172.21.15.83:0/4125156414 learned_addr learned my addr 172.21.15.83:0/4125156414 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8affd700 1 --2- 172.21.15.83:0/4125156414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b8570 0x7f6e8c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8a7fc700 1 --2- 172.21.15.83:0/4125156414 >> v2:172.21.15.90:3300/0 conn(0x7f6e8c0d2880 0x7f6e8c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 -- 172.21.15.83:0/4125156414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b8570 msgr2=0x7f6e8c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 --2- 172.21.15.83:0/4125156414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b8570 0x7f6e8c06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 -- 172.21.15.83:0/4125156414 >> v2:172.21.15.90:3300/0 conn(0x7f6e8c0d2880 msgr2=0x7f6e8c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 --2- 172.21.15.83:0/4125156414 >> v2:172.21.15.90:3300/0 conn(0x7f6e8c0d2880 0x7f6e8c160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8b7fe700 1 -- 172.21.15.83:0/4125156414 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c0cd7c0 con 0x7f6e8c0b4940 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.143+0000 7f6e8a7fc700 1 --2- 172.21.15.83:0/4125156414 >> v2:172.21.15.90:3300/0 conn(0x7f6e8c0d2880 0x7f6e8c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e8b7fe700 1 --2- 172.21.15.83:0/4125156414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b4940 0x7f6e8c0b4d90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b5ab0 tx=0x7f6e8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dadf11af6aadfad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e89ffb700 1 -- 172.21.15.83:0/4125156414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400e070 con 0x7f6e8c0b4940 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e89ffb700 1 -- 172.21.15.83:0/4125156414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8400b9c0 con 0x7f6e8c0b4940 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e89ffb700 1 -- 172.21.15.83:0/4125156414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400bd70 con 0x7f6e8c0b4940 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 -- 172.21.15.83:0/4125156414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b4940 msgr2=0x7f6e8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 --2- 172.21.15.83:0/4125156414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b4940 0x7f6e8c0b4d90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b5ab0 tx=0x7f6e8400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 -- 172.21.15.83:0/4125156414 shutdown_connections 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 --2- 172.21.15.83:0/4125156414 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b8570 0x7f6e8c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 --2- 172.21.15.83:0/4125156414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b4940 0x7f6e8c0b4d90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 --2- 172.21.15.83:0/4125156414 >> v2:172.21.15.90:3300/0 conn(0x7f6e8c0d2880 0x7f6e8c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 -- 172.21.15.83:0/4125156414 >> 172.21.15.83:0/4125156414 conn(0x7f6e8c0d30a0 msgr2=0x7f6e8c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 -- 172.21.15.83:0/4125156414 shutdown_connections 2023-11-28T11:11:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.144+0000 7f6e919a0700 1 -- 172.21.15.83:0/4125156414 wait complete. 2023-11-28T11:11:24.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.146+0000 7f6e919a0700 1 Processor -- start 2023-11-28T11:11:24.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.146+0000 7f6e919a0700 1 -- start start 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e919a0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b4940 0x7f6e8c0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e919a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 0x7f6e8c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e919a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e8c0d2880 0x7f6e8c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e919a0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0c9fa0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e919a0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0ca280 con 0x7f6e8c0d2880 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e919a0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c0ca560 con 0x7f6e8c0b4940 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 0x7f6e8c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 0x7f6e8c157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52604/0 (socket says 172.21.15.83:52604) 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 -- 172.21.15.83:0/463196606 learned_addr learned my addr 172.21.15.83:0/463196606 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8affd700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b4940 0x7f6e8c0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8b7fe700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e8c0d2880 0x7f6e8c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 -- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b4940 msgr2=0x7f6e8c0cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b4940 0x7f6e8c0cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 -- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e8c0d2880 msgr2=0x7f6e8c0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e8c0d2880 0x7f6e8c0cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.147+0000 7f6e8a7fc700 1 -- 172.21.15.83:0/463196606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e84009b30 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.148+0000 7f6e8a7fc700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 0x7f6e8c157750 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00e7e0 tx=0x7f6e7c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.148+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c004230 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.148+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c004390 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.148+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c161160 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.148+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0036a0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.148+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c0cdaa0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.149+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c0cf6e0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.149+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e7c022070 con 0x7f6e8c0b8570 2023-11-28T11:11:24.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.150+0000 7f6e73fff700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.150+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f6e7c085840 con 0x7f6e8c0b8570 2023-11-28T11:11:24.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.150+0000 7f6e8affd700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.150+0000 7f6e8affd700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c900 tx=0x7f6e80008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.154+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e7c051fc0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.378+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6e8c0742b0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.384+0000 7f6e73fff700 1 -- 172.21.15.83:0/463196606 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v29) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6e7c0555e0 con 0x7f6e8c0b8570 2023-11-28T11:11:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 msgr2=0x7f6e7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c900 tx=0x7f6e80008040 comp rx=0 tx=0).stop 2023-11-28T11:11:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 msgr2=0x7f6e8c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 0x7f6e8c157750 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00e7e0 tx=0x7f6e7c00eba0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 shutdown_connections 2023-11-28T11:11:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e8c0d2880 0x7f6e8c0cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6e8c0b4940 0x7f6e8c0cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6e780682f0 0x7f6e7806a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 --2- 172.21.15.83:0/463196606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e8c0b8570 0x7f6e8c157750 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.388+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 >> 172.21.15.83:0/463196606 conn(0x7f6e8c0d30a0 msgr2=0x7f6e8c160900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.389+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 shutdown_connections 2023-11-28T11:11:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.389+0000 7f6e919a0700 1 -- 172.21.15.83:0/463196606 wait complete. 2023-11-28T11:11:24.398 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 10 and data pool 11 2023-11-28T11:11:24.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.534+0000 7f683c9a7700 1 Processor -- start 2023-11-28T11:11:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.534+0000 7f683c9a7700 1 -- start start 2023-11-28T11:11:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683c9a7700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f68380b88d0 0x7f68380cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683c9a7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380d0c50 con 0x7f6838160e60 2023-11-28T11:11:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683c9a7700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380cd760 con 0x7f68380b5170 2023-11-28T11:11:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683c9a7700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380b8ca0 con 0x7f68380b88d0 2023-11-28T11:11:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f6835d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6838160e60 0x7f68380b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50484/0 (socket says 172.21.15.83:50484) 2023-11-28T11:11:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f6835d9b700 1 -- 172.21.15.83:0/3034771766 learned_addr learned my addr 172.21.15.83:0/3034771766 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125333083 0 0) 0x7f68380d0c50 con 0x7f6838160e60 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683659c700 1 --2- 172.21.15.83:0/3034771766 >> v2:172.21.15.90:3300/0 conn(0x7f68380b88d0 0x7f68380cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.535+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68200035b0 con 0x7f6838160e60 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383279950 0 0) 0x7f68200035b0 con 0x7f6838160e60 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 >> v2:172.21.15.90:3300/0 conn(0x7f68380b88d0 msgr2=0x7f68380cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 --2- 172.21.15.83:0/3034771766 >> v2:172.21.15.90:3300/0 conn(0x7f68380b88d0 0x7f68380cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 >> v1:172.21.15.90:6789/0 conn(0x7f68380b5170 legacy=0x7f68380d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380cda40 con 0x7f6838160e60 2023-11-28T11:11:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f6828001670 con 0x7f6838160e60 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 >> v1:172.21.15.83:6789/0 conn(0x7f6838160e60 legacy=0x7f68380b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6820003a00 0x7f6820003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6820004850 0x7f6820004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f682000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380b8ca0 con 0x7f6820005200 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682000c110 con 0x7f6820004850 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682000c2d0 con 0x7f6820003a00 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f682000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683659c700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6820003a00 0x7f6820003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6835d9b700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6820004850 0x7f6820004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 -- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6820003a00 msgr2=0x7f6820003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:24.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6820003a00 0x7f6820003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 -- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6820004850 msgr2=0x7f6820004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6820004850 0x7f6820004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 -- 172.21.15.83:0/3034771766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200035b0 con 0x7f6820005200 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 -- 172.21.15.83:0/3034771766 reap_dead start 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f6836d9d700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f682000ba10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f683000c160 tx=0x7f6820005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed44823324718141 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830001db0 con 0x7f6820005200 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683000ccc0 con 0x7f6820005200 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.536+0000 7f683559a700 1 -- 172.21.15.83:0/3034771766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830003c40 con 0x7f6820005200 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 -- 172.21.15.83:0/3034771766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 msgr2=0x7f682000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f682000ba10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f683000c160 tx=0x7f6820005880 comp rx=0 tx=0).stop 2023-11-28T11:11:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 -- 172.21.15.83:0/3034771766 shutdown_connections 2023-11-28T11:11:24.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 --2- 172.21.15.83:0/3034771766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f682000ba10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 -- 172.21.15.83:0/3034771766 >> 172.21.15.83:0/3034771766 conn(0x7f683806d030 msgr2=0x7f68380d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:24.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 -- 172.21.15.83:0/3034771766 shutdown_connections 2023-11-28T11:11:24.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.537+0000 7f683c9a7700 1 -- 172.21.15.83:0/3034771766 wait complete. 2023-11-28T11:11:24.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.538+0000 7f683c9a7700 1 Processor -- start 2023-11-28T11:11:24.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 -- start start 2023-11-28T11:11:24.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f68380c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68381533b0 0x7f6838159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6838153820 0x7f6838159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381f5870 con 0x7f6820005200 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381f5b10 con 0x7f68381533b0 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683c9a7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68381f5df0 con 0x7f6838153820 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f68380c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f68380c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52634/0 (socket says 172.21.15.83:52634) 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.539+0000 7f683659c700 1 -- 172.21.15.83:0/2009913834 learned_addr learned my addr 172.21.15.83:0/2009913834 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f6835d9b700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68381533b0 0x7f6838159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f6836d9d700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6838153820 0x7f6838159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683659c700 1 -- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6838153820 msgr2=0x7f6838159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683659c700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6838153820 0x7f6838159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683659c700 1 -- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68381533b0 msgr2=0x7f6838159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683659c700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68381533b0 0x7f6838159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683659c700 1 -- 172.21.15.83:0/2009913834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68381612d0 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683659c700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f68380c8f00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f682c0119a0 tx=0x7f682c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00f940 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c00faa0 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00fe30 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380d0ef0 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.540+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68380cdd20 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.541+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6838074320 con 0x7f6820005200 2023-11-28T11:11:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.541+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f682c014440 con 0x7f6820005200 2023-11-28T11:11:24.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.542+0000 7f6826ffd700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f68180682f0 0x7f681806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.542+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f682c086700 con 0x7f6820005200 2023-11-28T11:11:24.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.542+0000 7f6835d9b700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f68180682f0 0x7f681806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.542+0000 7f6835d9b700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f68180682f0 0x7f681806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f683815a530 tx=0x7f6828009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.546+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f682c052cd0 con 0x7f6820005200 2023-11-28T11:11:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.731+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f683806b600 con 0x7f6820005200 2023-11-28T11:11:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.732+0000 7f6826ffd700 1 -- 172.21.15.83:0/2009913834 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18090 (secure 0 0 0) 0x7f682c0562f0 con 0x7f6820005200 2023-11-28T11:11:24.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:24.733 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:11:24.379100+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f68180682f0 msgr2=0x7f681806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f68180682f0 0x7f681806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f683815a530 tx=0x7f6828009040 comp rx=0 tx=0).stop 2023-11-28T11:11:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 msgr2=0x7f68380c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f68380c8f00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f682c0119a0 tx=0x7f682c011d60 comp rx=0 tx=0).stop 2023-11-28T11:11:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 shutdown_connections 2023-11-28T11:11:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6838153820 0x7f6838159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f68180682f0 0x7f681806a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820005200 0x7f68380c8f00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 --2- 172.21.15.83:0/2009913834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68381533b0 0x7f6838159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 >> 172.21.15.83:0/2009913834 conn(0x7f683806d030 msgr2=0x7f68380b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 shutdown_connections 2023-11-28T11:11:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.734+0000 7f683c9a7700 1 -- 172.21.15.83:0/2009913834 wait complete. 2023-11-28T11:11:24.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.867+0000 7f418b73d700 1 Processor -- start 2023-11-28T11:11:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.868+0000 7f418b73d700 1 -- start start 2023-11-28T11:11:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.868+0000 7f418b73d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f418415cd10 0x7f41840ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.868+0000 7f418b73d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840cbfb0 con 0x7f41840b1fb0 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.868+0000 7f418b73d700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840cb710 con 0x7f418415d0e0 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.868+0000 7f418b73d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41840cb9f0 con 0x7f418415cd10 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f418415cd10 0x7f41840ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f418415cd10 0x7f41840ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52642/0 (socket says 172.21.15.83:52642) 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f4189cda700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f41840b1fb0 0x7f418406bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50506/0 (socket says 172.21.15.83:50506) 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 -- 172.21.15.83:0/483798790 learned_addr learned my addr 172.21.15.83:0/483798790 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 -- 172.21.15.83:0/483798790 >> v1:172.21.15.90:6789/0 conn(0x7f418415d0e0 legacy=0x7f41840b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 -- 172.21.15.83:0/483798790 >> v1:172.21.15.83:6789/0 conn(0x7f41840b1fb0 legacy=0x7f418406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 -- 172.21.15.83:0/483798790 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840cbcd0 con 0x7f418415cd10 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f41894d9700 1 --2- 172.21.15.83:0/483798790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415cd10 0x7f41840ca970 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f41740098f0 tx=0x7f4174009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba58077a6948a53e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f417bfff700 1 -- 172.21.15.83:0/483798790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400b8a0 con 0x7f418415cd10 2023-11-28T11:11:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f417bfff700 1 -- 172.21.15.83:0/483798790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400ba00 con 0x7f418415cd10 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.869+0000 7f417bfff700 1 -- 172.21.15.83:0/483798790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400bd90 con 0x7f418415cd10 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 -- 172.21.15.83:0/483798790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415cd10 msgr2=0x7f41840ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 --2- 172.21.15.83:0/483798790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415cd10 0x7f41840ca970 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f41740098f0 tx=0x7f4174009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 -- 172.21.15.83:0/483798790 shutdown_connections 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 --2- 172.21.15.83:0/483798790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415cd10 0x7f41840ca970 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 -- 172.21.15.83:0/483798790 >> 172.21.15.83:0/483798790 conn(0x7f418406e9a0 msgr2=0x7f41840cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 -- 172.21.15.83:0/483798790 shutdown_connections 2023-11-28T11:11:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.870+0000 7f418b73d700 1 -- 172.21.15.83:0/483798790 wait complete. 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.871+0000 7f418b73d700 1 Processor -- start 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.871+0000 7f418b73d700 1 -- start start 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f41840b1fb0 0x7f4184151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f418415cd10 0x7f41840c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 0x7f41840c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41841e9410 con 0x7f418415d0e0 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41840c9d60 con 0x7f418415cd10 2023-11-28T11:11:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41840c4120 con 0x7f41840b1fb0 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 0x7f41840c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 0x7f41840c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52652/0 (socket says 172.21.15.83:52652) 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 -- 172.21.15.83:0/657731225 learned_addr learned my addr 172.21.15.83:0/657731225 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4188cd8700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f418415cd10 0x7f41840c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f41894d9700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f41840b1fb0 0x7f4184151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 -- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f41840b1fb0 msgr2=0x7f4184151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f41840b1fb0 0x7f4184151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 -- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f418415cd10 msgr2=0x7f41840c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f418415cd10 0x7f41840c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 -- 172.21.15.83:0/657731225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174009550 con 0x7f418415d0e0 2023-11-28T11:11:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4189cda700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 0x7f41840c7430 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f417c00cba0 tx=0x7f417c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00aaa0 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c00ac00 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418406b660 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.872+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c007070 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.873+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418406b9b0 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.873+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41840cd9f0 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.874+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f417c019030 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.874+0000 7f4179ffb700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f41680682f0 0x7f416806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.874+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f417c0859a0 con 0x7f418415d0e0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.874+0000 7f41894d9700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f41680682f0 0x7f416806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.875+0000 7f41894d9700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f41680682f0 0x7f416806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4174009c40 tx=0x7f4174008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:24.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:24.879+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f417c04b5e0 con 0x7f418415d0e0 2023-11-28T11:11:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.064+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4184074320 con 0x7f418415d0e0 2023-11-28T11:11:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.066+0000 7f4179ffb700 1 -- 172.21.15.83:0/657731225 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18090 (secure 0 0 0) 0x7f417c017090 con 0x7f418415d0e0 2023-11-28T11:11:25.066 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:11:24.379100+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f41680682f0 msgr2=0x7f416806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f41680682f0 0x7f416806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4174009c40 tx=0x7f4174008030 comp rx=0 tx=0).stop 2023-11-28T11:11:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 msgr2=0x7f41840c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 0x7f41840c7430 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f417c00cba0 tx=0x7f417c00ceb0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 shutdown_connections 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f41680682f0 0x7f416806a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f418415d0e0 0x7f41840c7430 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f41840b1fb0 0x7f4184151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 --2- 172.21.15.83:0/657731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f418415cd10 0x7f41840c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 >> 172.21.15.83:0/657731225 conn(0x7f418406e9a0 msgr2=0x7f41840ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 shutdown_connections 2023-11-28T11:11:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.068+0000 7f418b73d700 1 -- 172.21.15.83:0/657731225 wait complete. 2023-11-28T11:11:25.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:11:25.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.203+0000 7f87e2055700 1 Processor -- start 2023-11-28T11:11:25.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.203+0000 7f87e2055700 1 -- start start 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87e2055700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f87dc158a40 0x7f87dc0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87e2055700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87e2055700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc0cd940 con 0x7f87dc158e10 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87e2055700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0ca540 con 0x7f87dc158a40 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87e2055700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0cdc20 con 0x7f87dc071fa0 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87d2ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f87dc158e10 0x7f87dc071890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50518/0 (socket says 172.21.15.83:50518) 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87d2ffd700 1 -- 172.21.15.83:0/3727132535 learned_addr learned my addr 172.21.15.83:0/3727132535 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87dbfff700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.204+0000 7f87db7fe700 1 --2- 172.21.15.83:0/3727132535 >> v2:172.21.15.90:3300/0 conn(0x7f87dc158a40 0x7f87dc0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87daffd700 1 -- 172.21.15.83:0/3727132535 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654261990 0 0) 0x7f87dc0cd940 con 0x7f87dc158e10 2023-11-28T11:11:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87daffd700 1 -- 172.21.15.83:0/3727132535 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c80035b0 con 0x7f87dc158e10 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87db7fe700 1 -- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 msgr2=0x7f87dc0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87db7fe700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87db7fe700 1 -- 172.21.15.83:0/3727132535 >> v1:172.21.15.83:6789/0 conn(0x7f87dc158e10 legacy=0x7f87dc071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87daffd700 1 -- 172.21.15.83:0/3727132535 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3519418013 0 0) 0x7f87c80035b0 con 0x7f87dc158e10 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87db7fe700 1 -- 172.21.15.83:0/3727132535 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0cdf00 con 0x7f87dc158a40 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87dbfff700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87db7fe700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc0ce9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009840 tx=0x7f87cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a67e12dca41fe09 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87daffd700 1 -- 172.21.15.83:0/3727132535 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4001670 con 0x7f87dc158a40 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87daffd700 1 -- 172.21.15.83:0/3727132535 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc004030 con 0x7f87dc158a40 2023-11-28T11:11:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.205+0000 7f87daffd700 1 -- 172.21.15.83:0/3727132535 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0043c0 con 0x7f87dc158a40 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 -- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 msgr2=0x7f87dc0ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc0ce9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009840 tx=0x7f87cc009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 -- 172.21.15.83:0/3727132535 shutdown_connections 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 --2- 172.21.15.83:0/3727132535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc0ce9e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 -- 172.21.15.83:0/3727132535 >> 172.21.15.83:0/3727132535 conn(0x7f87dc0cba50 msgr2=0x7f87dc0cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 -- 172.21.15.83:0/3727132535 shutdown_connections 2023-11-28T11:11:25.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.206+0000 7f87e2055700 1 -- 172.21.15.83:0/3727132535 wait complete. 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.207+0000 7f87e2055700 1 Processor -- start 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.207+0000 7f87e2055700 1 -- start start 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87e2055700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc1e1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87e2055700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87e2055700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 0x7f87dc1f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87e2055700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1515c0 con 0x7f87dc158e10 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87e2055700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc15a440 con 0x7f87dc158a40 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87e2055700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1e4060 con 0x7f87dc071fa0 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 0x7f87dc1f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 0x7f87dc1f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52664/0 (socket says 172.21.15.83:52664) 2023-11-28T11:11:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 -- 172.21.15.83:0/3662425970 learned_addr learned my addr 172.21.15.83:0/3662425970 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87db7fe700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc1e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87d2ffd700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 -- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 msgr2=0x7f87dc1e1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc1e1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 -- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 msgr2=0x7f87dc151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.208+0000 7f87dbfff700 1 -- 172.21.15.83:0/3662425970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4009a70 con 0x7f87dc158e10 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.209+0000 7f87dbfff700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 0x7f87dc1f5830 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f87d400efc0 tx=0x7f87d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.209+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400ca90 con 0x7f87dc158e10 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.209+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400f040 con 0x7f87dc158e10 2023-11-28T11:11:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.209+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc0094a0 con 0x7f87dc158e10 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.209+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400ce10 con 0x7f87dc158e10 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.209+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0ce4c0 con 0x7f87dc158e10 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.210+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc0ce1e0 con 0x7f87dc158e10 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.210+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87d4014480 con 0x7f87dc158e10 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.211+0000 7f87d8ff9700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87bc0682f0 0x7f87bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.211+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f87d4086540 con 0x7f87dc158e10 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.211+0000 7f87db7fe700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87bc0682f0 0x7f87bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:25.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.211+0000 7f87db7fe700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87bc0682f0 0x7f87bc06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0090b0 tx=0x7f87cc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:25.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.215+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87d4052c00 con 0x7f87dc158e10 2023-11-28T11:11:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:25.442+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f87dc0cd150 con 0x7f87dc158e10 2023-11-28T11:11:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.391+0000 7f87d8ff9700 1 -- 172.21.15.83:0/3662425970 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f87d4009d70 con 0x7f87dc158e10 2023-11-28T11:11:26.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87bc0682f0 msgr2=0x7f87bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:26.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87bc0682f0 0x7f87bc06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0090b0 tx=0x7f87cc008030 comp rx=0 tx=0).stop 2023-11-28T11:11:26.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 msgr2=0x7f87dc1f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 0x7f87dc1f5830 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f87d400efc0 tx=0x7f87d400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 shutdown_connections 2023-11-28T11:11:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f87bc0682f0 0x7f87bc06a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc158e10 0x7f87dc1f5830 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87dc071fa0 0x7f87dc1e1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 --2- 172.21.15.83:0/3662425970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87dc158a40 0x7f87dc151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.393+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 >> 172.21.15.83:0/3662425970 conn(0x7f87dc0cba50 msgr2=0x7f87dc0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:26.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.394+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 shutdown_connections 2023-11-28T11:11:26.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.394+0000 7f87e2055700 1 -- 172.21.15.83:0/3662425970 wait complete. 2023-11-28T11:11:26.409 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:11:26.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:11:26.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.537+0000 7f7453a6d700 1 Processor -- start 2023-11-28T11:11:26.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.537+0000 7f7453a6d700 1 -- start start 2023-11-28T11:11:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7453a6d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f744c15cd30 0x7f744c0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7453a6d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0cb200 0x7f744c0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7453a6d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c06cad0 con 0x7f744c0b1d80 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7453a6d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744c0cb670 con 0x7f744c15cd30 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7453a6d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c0af4a0 con 0x7f744c0cb200 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f744c15cd30 0x7f744c0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f744c15cd30 0x7f744c0cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52670/0 (socket says 172.21.15.83:52670) 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 -- 172.21.15.83:0/1117371584 learned_addr learned my addr 172.21.15.83:0/1117371584 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f745200a700 1 --1- 172.21.15.83:0/1117371584 >> v1:172.21.15.83:6789/0 conn(0x7f744c0b1d80 0x7f744c1606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50534/0 (socket says 172.21.15.83:50534) 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451008700 1 --2- 172.21.15.83:0/1117371584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0cb200 0x7f744c0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 -- 172.21.15.83:0/1117371584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0cb200 msgr2=0x7f744c0b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 --2- 172.21.15.83:0/1117371584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0cb200 0x7f744c0b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7450807700 1 -- 172.21.15.83:0/1117371584 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377077258 0 0) 0x7f744c06cad0 con 0x7f744c0b1d80 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 -- 172.21.15.83:0/1117371584 >> v1:172.21.15.83:6789/0 conn(0x7f744c0b1d80 legacy=0x7f744c1606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 -- 172.21.15.83:0/1117371584 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0ce7f0 con 0x7f744c15cd30 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7451809700 1 --2- 172.21.15.83:0/1117371584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0cacc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f74480098f0 tx=0x7f7448009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85730914711fd5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7450807700 1 -- 172.21.15.83:0/1117371584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800b8a0 con 0x7f744c15cd30 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7450807700 1 -- 172.21.15.83:0/1117371584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744800ba00 con 0x7f744c15cd30 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.538+0000 7f7450807700 1 -- 172.21.15.83:0/1117371584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800bd90 con 0x7f744c15cd30 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 -- 172.21.15.83:0/1117371584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 msgr2=0x7f744c0cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 --2- 172.21.15.83:0/1117371584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0cacc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f74480098f0 tx=0x7f7448009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 -- 172.21.15.83:0/1117371584 shutdown_connections 2023-11-28T11:11:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 --2- 172.21.15.83:0/1117371584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0cacc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 --2- 172.21.15.83:0/1117371584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0cb200 0x7f744c0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 -- 172.21.15.83:0/1117371584 >> 172.21.15.83:0/1117371584 conn(0x7f744c0cc690 msgr2=0x7f744c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 -- 172.21.15.83:0/1117371584 shutdown_connections 2023-11-28T11:11:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.539+0000 7f7453a6d700 1 -- 172.21.15.83:0/1117371584 wait complete. 2023-11-28T11:11:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.540+0000 7f7453a6d700 1 Processor -- start 2023-11-28T11:11:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.540+0000 7f7453a6d700 1 -- start start 2023-11-28T11:11:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7453a6d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0b1d80 0x7f744c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7453a6d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c0cb200 0x7f744c0b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7453a6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7453a6d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c0c67a0 con 0x7f744c15cd30 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7453a6d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c15d5a0 con 0x7f744c0cb200 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7453a6d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c0c9b70 con 0x7f744c0b1d80 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52680/0 (socket says 172.21.15.83:52680) 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 -- 172.21.15.83:0/19198790 learned_addr learned my addr 172.21.15.83:0/19198790 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7451809700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0b1d80 0x7f744c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7451008700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c0cb200 0x7f744c0b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 -- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0b1d80 msgr2=0x7f744c0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0b1d80 0x7f744c0c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 -- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c0cb200 msgr2=0x7f744c0b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c0cb200 0x7f744c0b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 -- 172.21.15.83:0/19198790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448009550 con 0x7f744c15cd30 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f7451008700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c0cb200 0x7f744c0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.541+0000 7f745200a700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0b5170 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f744400cba0 tx=0x7f744400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.542+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400aad0 con 0x7f744c15cd30 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.542+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744400ac50 con 0x7f744c15cd30 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.542+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c06c1a0 con 0x7f744c15cd30 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.542+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440070d0 con 0x7f744c15cd30 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.542+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c06bec0 con 0x7f744c15cd30 2023-11-28T11:11:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.543+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c06c480 con 0x7f744c15cd30 2023-11-28T11:11:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.543+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7444010070 con 0x7f744c15cd30 2023-11-28T11:11:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.543+0000 7f74427fc700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74300682f0 0x7f743006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.544+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f74440856e0 con 0x7f744c15cd30 2023-11-28T11:11:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.544+0000 7f7451809700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74300682f0 0x7f743006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.544+0000 7f7451809700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74300682f0 0x7f743006a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7448009c40 tx=0x7f7448008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.548+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7444051da0 con 0x7f744c15cd30 2023-11-28T11:11:26.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:26.774+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f744c15d6e0 con 0x7f744c15cd30 2023-11-28T11:11:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.391+0000 7f74427fc700 1 -- 172.21.15.83:0/19198790 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74440553c0 con 0x7f744c15cd30 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74300682f0 msgr2=0x7f743006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74300682f0 0x7f743006a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7448009c40 tx=0x7f7448008030 comp rx=0 tx=0).stop 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 msgr2=0x7f744c0b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0b5170 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f744400cba0 tx=0x7f744400ceb0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 shutdown_connections 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f74300682f0 0x7f743006a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f744c15cd30 0x7f744c0b5170 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f744c0b1d80 0x7f744c0c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 --2- 172.21.15.83:0/19198790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c0cb200 0x7f744c0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 >> 172.21.15.83:0/19198790 conn(0x7f744c0cc690 msgr2=0x7f744c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 shutdown_connections 2023-11-28T11:11:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.393+0000 7f7453a6d700 1 -- 172.21.15.83:0/19198790 wait complete. 2023-11-28T11:11:27.407 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:11:27.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.526+0000 7ff78432d700 1 Processor -- start 2023-11-28T11:11:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.526+0000 7ff78432d700 1 -- start start 2023-11-28T11:11:27.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff78432d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff78432d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff77c06fdb0 0x7ff77c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff78432d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0b0790 con 0x7ff77c0cddd0 2023-11-28T11:11:27.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff78432d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c06caa0 con 0x7ff77c06fdb0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff78432d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15cce0 con 0x7ff77c0cf1c0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff7828ca700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff77c06fdb0 0x7ff77c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.527+0000 7ff7820c9700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff77c0cddd0 0x7ff77c0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50544/0 (socket says 172.21.15.83:50544) 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7820c9700 1 -- 172.21.15.83:0/3363011750 learned_addr learned my addr 172.21.15.83:0/3363011750 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7818c8700 1 --2- 172.21.15.83:0/3363011750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7828ca700 1 -- 172.21.15.83:0/3363011750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 msgr2=0x7ff77c160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7828ca700 1 --2- 172.21.15.83:0/3363011750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7828ca700 1 -- 172.21.15.83:0/3363011750 >> v1:172.21.15.83:6789/0 conn(0x7ff77c0cddd0 legacy=0x7ff77c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7828ca700 1 -- 172.21.15.83:0/3363011750 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c06be20 con 0x7ff77c06fdb0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7810c7700 1 -- 172.21.15.83:0/3363011750 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645005071 0 0) 0x7ff77c0b0790 con 0x7ff77c0cddd0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7828ca700 1 --2- 172.21.15.83:0/3363011750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c0cb530 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff774009d90 tx=0x7ff77400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36b16464977c35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7810c7700 1 -- 172.21.15.83:0/3363011750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400e070 con 0x7ff77c06fdb0 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7810c7700 1 -- 172.21.15.83:0/3363011750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77400bbd0 con 0x7ff77c06fdb0 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.528+0000 7ff7810c7700 1 -- 172.21.15.83:0/3363011750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774012430 con 0x7ff77c06fdb0 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 -- 172.21.15.83:0/3363011750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 msgr2=0x7ff77c0cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 --2- 172.21.15.83:0/3363011750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c0cb530 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff774009d90 tx=0x7ff77400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 -- 172.21.15.83:0/3363011750 shutdown_connections 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 --2- 172.21.15.83:0/3363011750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c0cb530 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 --2- 172.21.15.83:0/3363011750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 -- 172.21.15.83:0/3363011750 >> 172.21.15.83:0/3363011750 conn(0x7ff77c06de20 msgr2=0x7ff77c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 -- 172.21.15.83:0/3363011750 shutdown_connections 2023-11-28T11:11:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.529+0000 7ff78432d700 1 -- 172.21.15.83:0/3363011750 wait complete. 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.530+0000 7ff78432d700 1 Processor -- start 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.530+0000 7ff78432d700 1 -- start start 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff77c0cddd0 0x7ff77c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0c4a80 con 0x7ff77c06fdb0 2023-11-28T11:11:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0c4d60 con 0x7ff77c0cddd0 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15e480 con 0x7ff77c0cf1c0 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52690/0 (socket says 172.21.15.83:52690) 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 -- 172.21.15.83:0/1790707101 learned_addr learned my addr 172.21.15.83:0/1790707101 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7828ca700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7818c8700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff77c0cddd0 0x7ff77c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 -- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 msgr2=0x7ff77c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c15ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 -- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff77c0cddd0 msgr2=0x7ff77c15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff77c0cddd0 0x7ff77c15a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 -- 172.21.15.83:0/1790707101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009a70 con 0x7ff77c06fdb0 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7818c8700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff77c0cddd0 0x7ff77c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:27.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7828ca700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff7820c9700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c15df40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0011e0 tx=0x7ff76c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004c80 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c004e00 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c06c3e0 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007310 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.531+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c06c100 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.532+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c06c6c0 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.533+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff76c01a020 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.533+0000 7ff772ffd700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff7600682f0 0x7ff76006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.533+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff76c085a50 con 0x7ff77c06fdb0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.533+0000 7ff7818c8700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff7600682f0 0x7ff76006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.533+0000 7ff7818c8700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff7600682f0 0x7ff76006a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff77c15ff80 tx=0x7ff778008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:27.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.537+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff76c052170 con 0x7ff77c06fdb0 2023-11-28T11:11:27.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.760+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff77c074320 con 0x7ff77c06fdb0 2023-11-28T11:11:27.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.761+0000 7ff772ffd700 1 -- 172.21.15.83:0/1790707101 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7ff76c055790 con 0x7ff77c06fdb0 2023-11-28T11:11:27.763 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:27.389584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff7600682f0 msgr2=0x7ff76006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff7600682f0 0x7ff76006a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff77c15ff80 tx=0x7ff778008040 comp rx=0 tx=0).stop 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 msgr2=0x7ff77c15df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c15df40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0011e0 tx=0x7ff76c00cae0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 shutdown_connections 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff77c0cf1c0 0x7ff77c15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff7600682f0 0x7ff76006a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff77c06fdb0 0x7ff77c15df40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 --2- 172.21.15.83:0/1790707101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff77c0cddd0 0x7ff77c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 >> 172.21.15.83:0/1790707101 conn(0x7ff77c06de20 msgr2=0x7ff77c15bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 shutdown_connections 2023-11-28T11:11:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.763+0000 7ff78432d700 1 -- 172.21.15.83:0/1790707101 wait complete. 2023-11-28T11:11:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2023-11-28T11:11:27.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.895+0000 7f9578c4f700 1 Processor -- start 2023-11-28T11:11:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.895+0000 7f9578c4f700 1 -- start start 2023-11-28T11:11:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.896+0000 7f9578c4f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f957406bfc0 0x7f957406af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.896+0000 7f9578c4f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95740cb220 con 0x7f957406ab00 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.896+0000 7f9578c4f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957406c400 con 0x7f957415aad0 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.896+0000 7f9578c4f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957406b4c0 con 0x7f957406bfc0 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9571d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f957406ab00 0x7f957406b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50560/0 (socket says 172.21.15.83:50560) 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9571d9b700 1 -- 172.21.15.83:0/3250850809 learned_addr learned my addr 172.21.15.83:0/3250850809 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9572d9d700 1 --2- 172.21.15.83:0/3250850809 >> v2:172.21.15.90:3300/0 conn(0x7f957406bfc0 0x7f957406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f957159a700 1 -- 172.21.15.83:0/3250850809 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648340060 0 0) 0x7f95740cb220 con 0x7f957406ab00 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f957159a700 1 -- 172.21.15.83:0/3250850809 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f955c0035b0 con 0x7f957406ab00 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9572d9d700 1 -- 172.21.15.83:0/3250850809 >> v1:172.21.15.90:6789/0 conn(0x7f957415aad0 legacy=0x7f95740b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9572d9d700 1 -- 172.21.15.83:0/3250850809 >> v1:172.21.15.83:6789/0 conn(0x7f957406ab00 legacy=0x7f957406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f957159a700 1 -- 172.21.15.83:0/3250850809 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265862773 0 0) 0x7f955c0035b0 con 0x7f957406ab00 2023-11-28T11:11:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9572d9d700 1 -- 172.21.15.83:0/3250850809 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957415aea0 con 0x7f957406bfc0 2023-11-28T11:11:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f9572d9d700 1 --2- 172.21.15.83:0/3250850809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957406bfc0 0x7f957406af80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f956c009d90 tx=0x7f956c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=593737020b1d4eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f957159a700 1 -- 172.21.15.83:0/3250850809 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c00e070 con 0x7f957406bfc0 2023-11-28T11:11:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f957159a700 1 -- 172.21.15.83:0/3250850809 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c003a20 con 0x7f957406bfc0 2023-11-28T11:11:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.897+0000 7f957159a700 1 -- 172.21.15.83:0/3250850809 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564001670 con 0x7f957406bfc0 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 -- 172.21.15.83:0/3250850809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957406bfc0 msgr2=0x7f957406af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 --2- 172.21.15.83:0/3250850809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957406bfc0 0x7f957406af80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f956c009d90 tx=0x7f956c00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 -- 172.21.15.83:0/3250850809 shutdown_connections 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 --2- 172.21.15.83:0/3250850809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957406bfc0 0x7f957406af80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 -- 172.21.15.83:0/3250850809 >> 172.21.15.83:0/3250850809 conn(0x7f957406e9a0 msgr2=0x7f95740cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 -- 172.21.15.83:0/3250850809 shutdown_connections 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.898+0000 7f9578c4f700 1 -- 172.21.15.83:0/3250850809 wait complete. 2023-11-28T11:11:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.899+0000 7f9578c4f700 1 Processor -- start 2023-11-28T11:11:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.899+0000 7f9578c4f700 1 -- start start 2023-11-28T11:11:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9578c4f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 0x7f95740c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9578c4f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f957406bfc0 0x7f95741545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9578c4f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957415aad0 0x7f95740b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9578c4f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95740c7430 con 0x7f957406ab00 2023-11-28T11:11:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9578c4f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9574154af0 con 0x7f957415aad0 2023-11-28T11:11:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9578c4f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9574157700 con 0x7f957406bfc0 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f957259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 0x7f95740c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f957259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 0x7f95740c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52704/0 (socket says 172.21.15.83:52704) 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f957259c700 1 -- 172.21.15.83:0/1047773331 learned_addr learned my addr 172.21.15.83:0/1047773331 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9572d9d700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957415aad0 0x7f95740b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.900+0000 7f9571d9b700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f957406bfc0 0x7f95741545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f957259c700 1 -- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f957406bfc0 msgr2=0x7f95741545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f957259c700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f957406bfc0 0x7f95741545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f957259c700 1 -- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957415aad0 msgr2=0x7f95740b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f957259c700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957415aad0 0x7f95740b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f957259c700 1 -- 172.21.15.83:0/1047773331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c009a70 con 0x7f957406ab00 2023-11-28T11:11:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f957259c700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 0x7f95740c6ef0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f956800c510 tx=0x7f956800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680052d0 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9568005430 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95740ca8f0 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680038c0 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.901+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95740ca610 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.902+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95740cabd0 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.902+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9568019030 con 0x7f957406ab00 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.903+0000 7f9562ffd700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f95540682f0 0x7f955406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.903+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9568085180 con 0x7f957406ab00 2023-11-28T11:11:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.903+0000 7f9571d9b700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f95540682f0 0x7f955406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.903+0000 7f9571d9b700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f95540682f0 0x7f955406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9564001eb0 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:27.907+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95680518a0 con 0x7f957406ab00 2023-11-28T11:11:28.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.092+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9574074320 con 0x7f957406ab00 2023-11-28T11:11:28.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.093+0000 7f9562ffd700 1 -- 172.21.15.83:0/1047773331 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18077 (secure 0 0 0) 0x7f9568054ec0 con 0x7f957406ab00 2023-11-28T11:11:28.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:28.095 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":63,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:11:25.381217+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f95540682f0 msgr2=0x7f955406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f95540682f0 0x7f955406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9564001eb0 tx=0x7f9564009040 comp rx=0 tx=0).stop 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 msgr2=0x7f95740c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 0x7f95740c6ef0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f956800c510 tx=0x7f956800c8d0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 shutdown_connections 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f957415aad0 0x7f95740b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f95540682f0 0x7f955406a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957406ab00 0x7f95740c6ef0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 --2- 172.21.15.83:0/1047773331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f957406bfc0 0x7f95741545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 >> 172.21.15.83:0/1047773331 conn(0x7f957406e9a0 msgr2=0x7f95740cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:28.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 shutdown_connections 2023-11-28T11:11:28.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.095+0000 7f9578c4f700 1 -- 172.21.15.83:0/1047773331 wait complete. 2023-11-28T11:11:28.109 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:11:28.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.229+0000 7f5a3380a700 1 Processor -- start 2023-11-28T11:11:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.229+0000 7f5a3380a700 1 -- start start 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a3380a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a2c0cf570 0x7f5a2c15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a3380a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c0ce6c0 con 0x7f5a2c0cd090 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a3380a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c0cd460 con 0x7f5a2c0b2510 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a3380a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2c159d50 con 0x7f5a2c0cf570 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a30da5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a2c0cf570 0x7f5a2c15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a315a6700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5a2c0cd090 0x7f5a2c0cee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50572/0 (socket says 172.21.15.83:50572) 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a315a6700 1 -- 172.21.15.83:0/1900883782 learned_addr learned my addr 172.21.15.83:0/1900883782 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a30da5700 1 -- 172.21.15.83:0/1900883782 >> v1:172.21.15.90:6789/0 conn(0x7f5a2c0b2510 legacy=0x7f5a2c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a30da5700 1 -- 172.21.15.83:0/1900883782 >> v1:172.21.15.83:6789/0 conn(0x7f5a2c0cd090 legacy=0x7f5a2c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a23fff700 1 -- 172.21.15.83:0/1900883782 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243381116 0 0) 0x7f5a2c0ce6c0 con 0x7f5a2c0cd090 2023-11-28T11:11:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a30da5700 1 -- 172.21.15.83:0/1900883782 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c0cdab0 con 0x7f5a2c0cf570 2023-11-28T11:11:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a30da5700 1 --2- 172.21.15.83:0/1900883782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c15ddb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009840 tx=0x7f5a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a97fa2ed505565 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.230+0000 7f5a23fff700 1 -- 172.21.15.83:0/1900883782 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c004230 con 0x7f5a2c0cf570 2023-11-28T11:11:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a23fff700 1 -- 172.21.15.83:0/1900883782 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c004390 con 0x7f5a2c0cf570 2023-11-28T11:11:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a23fff700 1 -- 172.21.15.83:0/1900883782 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0036a0 con 0x7f5a2c0cf570 2023-11-28T11:11:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 -- 172.21.15.83:0/1900883782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 msgr2=0x7f5a2c15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 --2- 172.21.15.83:0/1900883782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c15ddb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009840 tx=0x7f5a1c009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 -- 172.21.15.83:0/1900883782 shutdown_connections 2023-11-28T11:11:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 --2- 172.21.15.83:0/1900883782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c15ddb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 -- 172.21.15.83:0/1900883782 >> 172.21.15.83:0/1900883782 conn(0x7f5a2c06e9f0 msgr2=0x7f5a2c0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 -- 172.21.15.83:0/1900883782 shutdown_connections 2023-11-28T11:11:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.231+0000 7f5a3380a700 1 -- 172.21.15.83:0/1900883782 wait complete. 2023-11-28T11:11:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.233+0000 7f5a3380a700 1 Processor -- start 2023-11-28T11:11:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.233+0000 7f5a3380a700 1 -- start start 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0b2510 0x7f5a2c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c0cd090 0x7f5a2c150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c153540 con 0x7f5a2c0cf570 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c150fc0 con 0x7f5a2c0b2510 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c150ce0 con 0x7f5a2c0cd090 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c0c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52722/0 (socket says 172.21.15.83:52722) 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 -- 172.21.15.83:0/4254450072 learned_addr learned my addr 172.21.15.83:0/4254450072 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a30da5700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c0cd090 0x7f5a2c150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a315a6700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0b2510 0x7f5a2c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 -- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c0cd090 msgr2=0x7f5a2c150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c0cd090 0x7f5a2c150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 -- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0b2510 msgr2=0x7f5a2c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0b2510 0x7f5a2c153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 -- 172.21.15.83:0/4254450072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0094a0 con 0x7f5a2c0cf570 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a31da7700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c0c59f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c7a0 tx=0x7f5a2400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a24003e10 con 0x7f5a2c0cf570 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a24005070 con 0x7f5a2c0cf570 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c0ce070 con 0x7f5a2c0cf570 2023-11-28T11:11:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.234+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a24005420 con 0x7f5a2c0cf570 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.235+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c0cdd90 con 0x7f5a2c0cf570 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.235+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c0ce350 con 0x7f5a2c0cf570 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.236+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5a24019030 con 0x7f5a2c0cf570 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.236+0000 7f5a21ffb700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a100682f0 0x7f5a1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.236+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5a240853a0 con 0x7f5a2c0cf570 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.236+0000 7f5a315a6700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a100682f0 0x7f5a1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.236+0000 7f5a315a6700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a100682f0 0x7f5a1006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000f60 tx=0x7f5a28007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.240+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5a24051ac0 con 0x7f5a2c0cf570 2023-11-28T11:11:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.421+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a2c15aff0 con 0x7f5a100682f0 2023-11-28T11:11:28.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.429+0000 7f5a21ffb700 1 -- 172.21.15.83:0/4254450072 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300968 (secure 0 0 0) 0x7f5a2c15aff0 con 0x7f5a100682f0 2023-11-28T11:11:28.436 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:28.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a100682f0 msgr2=0x7f5a1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a100682f0 0x7f5a1006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000f60 tx=0x7f5a28007be0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 msgr2=0x7f5a2c0c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c0c59f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c7a0 tx=0x7f5a2400cb60 comp rx=0 tx=0).stop 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 shutdown_connections 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5a100682f0 0x7f5a1006a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2c0cf570 0x7f5a2c0c59f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0b2510 0x7f5a2c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 --2- 172.21.15.83:0/4254450072 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c0cd090 0x7f5a2c150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.432+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 >> 172.21.15.83:0/4254450072 conn(0x7f5a2c06e9f0 msgr2=0x7f5a2c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:28.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.433+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 shutdown_connections 2023-11-28T11:11:28.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.433+0000 7f5a3380a700 1 -- 172.21.15.83:0/4254450072 wait complete. 2023-11-28T11:11:28.441 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:11:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-28T11:11:27.605173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78024,"kb_used_data":6792,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":754896696,"statfs":{"total":773094113280,"available":773014216704,"internally_reserved":0,"allocated":6955008,"data_stored":3848314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619076"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.991933+0000","last_change":"2023-11-28T11:11:22.991933+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.991933+0000","last_undegraded":"2023-11-28T11:11:22.991933+0000","last_fullsized":"2023-11-28T11:11:22.991933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:51.009482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989322+0000","last_change":"2023-11-28T11:11:22.989322+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989322+0000","last_undegraded":"2023-11-28T11:11:22.989322+0000","last_fullsized":"2023-11-28T11:11:22.989322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:03.348424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990291+0000","last_change":"2023-11-28T11:11:22.990291+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990291+0000","last_undegraded":"2023-11-28T11:11:22.990291+0000","last_fullsized":"2023-11-28T11:11:22.990291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:12.037178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993692+0000","last_change":"2023-11-28T11:11:22.993692+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993692+0000","last_undegraded":"2023-11-28T11:11:22.993692+0000","last_fullsized":"2023-11-28T11:11:22.993692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:40.682923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.992400+0000","last_change":"2023-11-28T11:11:22.992400+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.992400+0000","last_undegraded":"2023-11-28T11:11:22.992400+0000","last_fullsized":"2023-11-28T11:11:22.992400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:08.956676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994804+0000","last_change":"2023-11-28T11:11:22.994804+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994804+0000","last_undegraded":"2023-11-28T11:11:22.994804+0000","last_fullsized":"2023-11-28T11:11:22.994804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:49.819742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987647+0000","last_change":"2023-11-28T11:11:22.987647+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987647+0000","last_undegraded":"2023-11-28T11:11:22.987647+0000","last_fullsized":"2023-11-28T11:11:22.987647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.988972+0000","last_change":"2023-11-28T11:11:22.988972+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.988972+0000","last_undegraded":"2023-11-28T11:11:22.988972+0000","last_fullsized":"2023-11-28T11:11:22.988972+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:53.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990325+0000","last_change":"2023-11-28T11:11:22.990325+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990325+0000","last_undegraded":"2023-11-28T11:11:22.990325+0000","last_fullsized":"2023-11-28T11:11:22.990325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:00.027787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993387+0000","last_change":"2023-11-28T11:11:22.993387+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993387+0000","last_undegraded":"2023-11-28T11:11:22.993387+0000","last_fullsized":"2023-11-28T11:11:22.993387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.992405+0000","last_change":"2023-11-28T11:11:22.992405+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.992405+0000","last_undegraded":"2023-11-28T11:11:22.992405+0000","last_fullsized":"2023-11-28T11:11:22.992405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:06.471882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987642+0000","last_change":"2023-11-28T11:11:22.987642+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987642+0000","last_undegraded":"2023-11-28T11:11:22.987642+0000","last_fullsized":"2023-11-28T11:11:22.987642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:03.658510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987726+0000","last_change":"2023-11-28T11:11:22.987726+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987726+0000","last_undegraded":"2023-11-28T11:11:22.987726+0000","last_fullsized":"2023-11-28T11:11:22.987726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994084+0000","last_change":"2023-11-28T11:11:22.994084+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994084+0000","last_undegraded":"2023-11-28T11:11:22.994084+0000","last_fullsized":"2023-11-28T11:11:22.994084+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:34.752567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987109+0000","last_change":"2023-11-28T11:11:22.987109+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987109+0000","last_undegraded":"2023-11-28T11:11:22.987109+0000","last_fullsized":"2023-11-28T11:11:22.987109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:21.311957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990182+0000","last_change":"2023-11-28T11:11:22.990182+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990182+0000","last_undegraded":"2023-11-28T11:11:22.990182+0000","last_fullsized":"2023-11-28T11:11:22.990182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:03.523669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993780+0000","last_change":"2023-11-28T11:11:22.993780+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993780+0000","last_undegraded":"2023-11-28T11:11:22.993780+0000","last_fullsized":"2023-11-28T11:11:22.993780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:20.035165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.986745+0000","last_change":"2023-11-28T11:11:22.986745+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.986745+0000","last_undegraded":"2023-11-28T11:11:22.986745+0000","last_fullsized":"2023-11-28T11:11:22.986745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:28.037670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989275+0000","last_change":"2023-11-28T11:11:22.989275+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989275+0000","last_undegraded":"2023-11-28T11:11:22.989275+0000","last_fullsized":"2023-11-28T11:11:22.989275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:43:12.693701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989274+0000","last_change":"2023-11-28T11:11:22.989274+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989274+0000","last_undegraded":"2023-11-28T11:11:22.989274+0000","last_fullsized":"2023-11-28T11:11:22.989274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:10.349250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.988966+0000","last_change":"2023-11-28T11:11:22.988966+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.988966+0000","last_undegraded":"2023-11-28T11:11:22.988966+0000","last_fullsized":"2023-11-28T11:11:22.988966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:36.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987636+0000","last_change":"2023-11-28T11:11:22.987636+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987636+0000","last_undegraded":"2023-11-28T11:11:22.987636+0000","last_fullsized":"2023-11-28T11:11:22.987636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990676+0000","last_change":"2023-11-28T11:11:22.990676+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990676+0000","last_undegraded":"2023-11-28T11:11:22.990676+0000","last_fullsized":"2023-11-28T11:11:22.990676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:26.554018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993143+0000","last_change":"2023-11-28T11:11:22.993143+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993143+0000","last_undegraded":"2023-11-28T11:11:22.993143+0000","last_fullsized":"2023-11-28T11:11:22.993143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:27.874016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990264+0000","last_change":"2023-11-28T11:11:22.990264+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990264+0000","last_undegraded":"2023-11-28T11:11:22.990264+0000","last_fullsized":"2023-11-28T11:11:22.990264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:09.661474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:20.094641+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:11:20.094641+0000","last_peered":"2023-11-28T11:11:20.094641+0000","last_clean":"2023-11-28T11:11:20.094641+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:11:20.094641+0000","last_undegraded":"2023-11-28T11:11:20.094641+0000","last_fullsized":"2023-11-28T11:11:20.094641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994667+0000","last_change":"2023-11-28T11:11:22.994667+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994667+0000","last_undegraded":"2023-11-28T11:11:22.994667+0000","last_fullsized":"2023-11-28T11:11:22.994667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.118149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.986707+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:11:21.986707+0000","last_peered":"2023-11-28T11:11:21.986707+0000","last_clean":"2023-11-28T11:11:21.986707+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:11:21.986707+0000","last_undegraded":"2023-11-28T11:11:21.986707+0000","last_fullsized":"2023-11-28T11:11:21.986707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989562+0000","last_change":"2023-11-28T11:11:22.989562+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989562+0000","last_undegraded":"2023-11-28T11:11:22.989562+0000","last_fullsized":"2023-11-28T11:11:22.989562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:42.594460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.399337+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:11:21.399337+0000","last_peered":"2023-11-28T11:11:21.399337+0000","last_clean":"2023-11-28T11:11:21.399337+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:11:21.399337+0000","last_undegraded":"2023-11-28T11:11:21.399337+0000","last_fullsized":"2023-11-28T11:11:21.399337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":194,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987549+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:11:21.987549+0000","last_peered":"2023-11-28T11:11:21.987549+0000","last_clean":"2023-11-28T11:11:21.987549+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:11:21.987549+0000","last_undegraded":"2023-11-28T11:11:21.987549+0000","last_fullsized":"2023-11-28T11:11:21.987549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.986627+0000","last_change":"2023-11-28T11:11:22.986627+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.986627+0000","last_undegraded":"2023-11-28T11:11:22.986627+0000","last_fullsized":"2023-11-28T11:11:22.986627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:01.938679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987547+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:11:21.987547+0000","last_peered":"2023-11-28T11:11:21.987547+0000","last_clean":"2023-11-28T11:11:21.987547+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:11:21.987547+0000","last_undegraded":"2023-11-28T11:11:21.987547+0000","last_fullsized":"2023-11-28T11:11:21.987547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993422+0000","last_change":"2023-11-28T11:11:22.993422+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993422+0000","last_undegraded":"2023-11-28T11:11:22.993422+0000","last_fullsized":"2023-11-28T11:11:22.993422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:25.947439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987514+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:11:21.987514+0000","last_peered":"2023-11-28T11:11:21.987514+0000","last_clean":"2023-11-28T11:11:21.987514+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:11:21.987514+0000","last_undegraded":"2023-11-28T11:11:21.987514+0000","last_fullsized":"2023-11-28T11:11:21.987514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993156+0000","last_change":"2023-11-28T11:11:22.993156+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993156+0000","last_undegraded":"2023-11-28T11:11:22.993156+0000","last_fullsized":"2023-11-28T11:11:22.993156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:20.094641+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:11:20.094641+0000","last_peered":"2023-11-28T11:11:20.094641+0000","last_clean":"2023-11-28T11:11:20.094641+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:11:20.094641+0000","last_undegraded":"2023-11-28T11:11:20.094641+0000","last_fullsized":"2023-11-28T11:11:20.094641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993454+0000","last_change":"2023-11-28T11:11:22.993454+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993454+0000","last_undegraded":"2023-11-28T11:11:22.993454+0000","last_fullsized":"2023-11-28T11:11:22.993454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987205+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:11:21.987205+0000","last_peered":"2023-11-28T11:11:21.987205+0000","last_clean":"2023-11-28T11:11:21.987205+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:11:21.987205+0000","last_undegraded":"2023-11-28T11:11:21.987205+0000","last_fullsized":"2023-11-28T11:11:21.987205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.991826+0000","last_change":"2023-11-28T11:11:22.991826+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.991826+0000","last_undegraded":"2023-11-28T11:11:22.991826+0000","last_fullsized":"2023-11-28T11:11:22.991826+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:44.971764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987232+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:11:21.987232+0000","last_peered":"2023-11-28T11:11:21.987232+0000","last_clean":"2023-11-28T11:11:21.987232+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:11:21.987232+0000","last_undegraded":"2023-11-28T11:11:21.987232+0000","last_fullsized":"2023-11-28T11:11:21.987232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10576,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361264,"statfs":{"total":96636764160,"available":96625934336,"internally_reserved":0,"allocated":1318912,"data_stored":925778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.376},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":708608,"data_stored":327559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":1.544,"15min":1.544},"last":0.59699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.337,"15min":1.337},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":1.665,"15min":1.665},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71399999999999997,"5min":1.762,"15min":1.762},"last":0.69899999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.60599999999999998,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72899999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71799999999999997,"5min":1.002,"15min":1.002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70399999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.877},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10512,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361328,"statfs":{"total":96636764160,"available":96625999872,"internally_reserved":0,"allocated":1318912,"data_stored":925778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38,"15min":0.39000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58699999999999997,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.504,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34499999999999997,"15min":0.34899999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.57399999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40999999999999998,"15min":0.41899999999999998},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.48499999999999999,"5min":1.363,"15min":1.363},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73799999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.752,"5min":1.54,"15min":1.54},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":1.48,"15min":1.48},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72899999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.755},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":1.004,"15min":1.004},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9660,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362180,"statfs":{"total":96636764160,"available":96626872320,"internally_reserved":0,"allocated":708608,"data_stored":327559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47999999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32500000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.378}]},{"osd":3,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40100000000000002},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.248},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58899999999999997,"5min":0.871,"15min":0.871},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72999999999999998,"5min":0.872,"15min":0.872},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.47399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9356,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362484,"statfs":{"total":96636764160,"available":96627183616,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.27000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.377},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.33400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78000000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75800000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80900000000000005,"5min":1.024,"15min":1.024},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83499999999999996,"5min":1.05,"15min":1.05},"last":0.625}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9612,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362228,"statfs":{"total":96636764160,"available":96626921472,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.499,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35799999999999998,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.219}]},{"osd":1,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":1.054,"15min":1.054},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53400000000000003,"5min":1.147,"15min":1.147},"last":0.251}]},{"osd":2,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.58699999999999997,"15min":0.622},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":5.133,"15min":5.133},"last":0.28100000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68100000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72599999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.194,"15min":1.194},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":1.077,"15min":1.077},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9612,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362228,"statfs":{"total":96636764160,"available":96626921472,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67400000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.246},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.27600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.29399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.747,"15min":0.747},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72499999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.30099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9420,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362420,"statfs":{"total":96636764160,"available":96627118080,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.81299999999999994,"15min":0.86099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84799999999999998,"15min":0.90000000000000002},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70999999999999996,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":6.601,"15min":6.601},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.375}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:28.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:28.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.605+0000 7f813cb4a700 1 Processor -- start 2023-11-28T11:11:28.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.605+0000 7f813cb4a700 1 -- start start 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813cb4a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 0x7f81380b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813cb4a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f813806bfc0 0x7f813806af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813cb4a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813806c440 con 0x7f813806ab00 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813cb4a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813806b4b0 con 0x7f813806bfc0 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813cb4a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380cb220 con 0x7f813815aad0 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f8135d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f813806ab00 0x7f813806b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50586/0 (socket says 172.21.15.83:50586) 2023-11-28T11:11:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f8135d9b700 1 -- 172.21.15.83:0/988503960 learned_addr learned my addr 172.21.15.83:0/988503960 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967883229 0 0) 0x7f813806c440 con 0x7f813806ab00 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f8136d9d700 1 --2- 172.21.15.83:0/988503960 >> v2:172.21.15.90:3300/0 conn(0x7f813806bfc0 0x7f813806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81200035b0 con 0x7f813806ab00 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813659c700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 0x7f81380b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401028742 0 0) 0x7f81200035b0 con 0x7f813806ab00 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 msgr2=0x7f81380b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 0x7f81380b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 >> v2:172.21.15.90:3300/0 conn(0x7f813806bfc0 msgr2=0x7f813806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 --2- 172.21.15.83:0/988503960 >> v2:172.21.15.90:3300/0 conn(0x7f813806bfc0 0x7f813806af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380b33d0 con 0x7f813806ab00 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f812c001670 con 0x7f813806ab00 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.606+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 >> v1:172.21.15.83:6789/0 conn(0x7f813806ab00 legacy=0x7f813806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 0x7f8120003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8120004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8120004d90 0x7f8120005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813806b4b0 con 0x7f8120003f80 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380cb220 con 0x7f8120004d90 2023-11-28T11:11:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812000ba40 con 0x7f813815aad0 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8120004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813659c700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 0x7f8120003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8136d9d700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8120004d90 0x7f8120005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 -- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 msgr2=0x7f8120003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f813815aad0 0x7f8120003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 -- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8120004d90 msgr2=0x7f8120005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8120004d90 0x7f8120005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 -- 172.21.15.83:0/988503960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200035b0 con 0x7f8120003f80 2023-11-28T11:11:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f8135d9b700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8120004850 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c370 tx=0x7f8120005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ec9e6c75791318 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c007c60 con 0x7f8120003f80 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c00ce50 con 0x7f8120003f80 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.607+0000 7f813559a700 1 -- 172.21.15.83:0/988503960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c005ba0 con 0x7f8120003f80 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f813cb4a700 1 -- 172.21.15.83:0/988503960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 msgr2=0x7f8120004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f813cb4a700 1 --2- 172.21.15.83:0/988503960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8120004850 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c370 tx=0x7f8120005880 comp rx=0 tx=0).stop 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f8136d9d700 1 -- 172.21.15.83:0/988503960 reap_dead start 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f813cb4a700 1 -- 172.21.15.83:0/988503960 shutdown_connections 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f813cb4a700 1 -- 172.21.15.83:0/988503960 >> 172.21.15.83:0/988503960 conn(0x7f813806e9a0 msgr2=0x7f81380cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f813cb4a700 1 -- 172.21.15.83:0/988503960 shutdown_connections 2023-11-28T11:11:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.608+0000 7f813cb4a700 1 -- 172.21.15.83:0/988503960 wait complete. 2023-11-28T11:11:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.609+0000 7f813cb4a700 1 Processor -- start 2023-11-28T11:11:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.609+0000 7f813cb4a700 1 -- start start 2023-11-28T11:11:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813cb4a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8138153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813cb4a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81381e5f00 0x7f81380cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813cb4a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81381e6370 0x7f81380cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813cb4a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138153bc0 con 0x7f8120003f80 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813cb4a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380cf960 con 0x7f81381e6370 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813cb4a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81381e5140 con 0x7f81381e5f00 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8138153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8138153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52738/0 (socket says 172.21.15.83:52738) 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 -- 172.21.15.83:0/3978828521 learned_addr learned my addr 172.21.15.83:0/3978828521 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f8136d9d700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81381e6370 0x7f81380cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f8135d9b700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81381e5f00 0x7f81380cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 -- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81381e5f00 msgr2=0x7f81380cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81381e5f00 0x7f81380cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 -- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81381e6370 msgr2=0x7f81380cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81381e6370 0x7f81380cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f813659c700 1 -- 172.21.15.83:0/3978828521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380ca610 con 0x7f8120003f80 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.610+0000 7f8135d9b700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81381e5f00 0x7f81380cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.611+0000 7f813659c700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8138153680 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8128011aa0 tx=0x7f8128011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.611+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800f940 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.611+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800faa0 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.611+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800fe50 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.611+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813815aea0 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.611+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81380ca8f0 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.612+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8138074320 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.612+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8128014440 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.612+0000 7f8126ffd700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81180682f0 0x7f811806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.613+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8128086670 con 0x7f8120003f80 2023-11-28T11:11:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.613+0000 7f8135d9b700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81180682f0 0x7f811806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.613+0000 7f8135d9b700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81180682f0 0x7f811806a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f812c015040 tx=0x7f812c00a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.617+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8128052e10 con 0x7f8120003f80 2023-11-28T11:11:28.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.794+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8138066560 con 0x7f81180682f0 2023-11-28T11:11:28.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.802+0000 7f8126ffd700 1 -- 172.21.15.83:0/3978828521 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300968 (secure 0 0 0) 0x7f8138066560 con 0x7f81180682f0 2023-11-28T11:11:28.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:28.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81180682f0 msgr2=0x7f811806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81180682f0 0x7f811806a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f812c015040 tx=0x7f812c00a000 comp rx=0 tx=0).stop 2023-11-28T11:11:28.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 msgr2=0x7f8138153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8138153680 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8128011aa0 tx=0x7f8128011db0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 shutdown_connections 2023-11-28T11:11:28.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81381e6370 0x7f81380cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81180682f0 0x7f811806a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120003f80 0x7f8138153680 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 --2- 172.21.15.83:0/3978828521 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81381e5f00 0x7f81380cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 >> 172.21.15.83:0/3978828521 conn(0x7f813806e9a0 msgr2=0x7f81380b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 shutdown_connections 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.805+0000 7f813cb4a700 1 -- 172.21.15.83:0/3978828521 wait complete. 2023-11-28T11:11:28.810 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:11:28.819 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-28T11:11:27.605173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78024,"kb_used_data":6792,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":754896696,"statfs":{"total":773094113280,"available":773014216704,"internally_reserved":0,"allocated":6955008,"data_stored":3848314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619076"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.991933+0000","last_change":"2023-11-28T11:11:22.991933+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.991933+0000","last_undegraded":"2023-11-28T11:11:22.991933+0000","last_fullsized":"2023-11-28T11:11:22.991933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:51.009482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989322+0000","last_change":"2023-11-28T11:11:22.989322+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989322+0000","last_undegraded":"2023-11-28T11:11:22.989322+0000","last_fullsized":"2023-11-28T11:11:22.989322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:03.348424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990291+0000","last_change":"2023-11-28T11:11:22.990291+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990291+0000","last_undegraded":"2023-11-28T11:11:22.990291+0000","last_fullsized":"2023-11-28T11:11:22.990291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:12.037178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993692+0000","last_change":"2023-11-28T11:11:22.993692+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993692+0000","last_undegraded":"2023-11-28T11:11:22.993692+0000","last_fullsized":"2023-11-28T11:11:22.993692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:40.682923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.992400+0000","last_change":"2023-11-28T11:11:22.992400+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.992400+0000","last_undegraded":"2023-11-28T11:11:22.992400+0000","last_fullsized":"2023-11-28T11:11:22.992400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:08.956676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994804+0000","last_change":"2023-11-28T11:11:22.994804+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994804+0000","last_undegraded":"2023-11-28T11:11:22.994804+0000","last_fullsized":"2023-11-28T11:11:22.994804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:49.819742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987647+0000","last_change":"2023-11-28T11:11:22.987647+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987647+0000","last_undegraded":"2023-11-28T11:11:22.987647+0000","last_fullsized":"2023-11-28T11:11:22.987647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.988972+0000","last_change":"2023-11-28T11:11:22.988972+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.988972+0000","last_undegraded":"2023-11-28T11:11:22.988972+0000","last_fullsized":"2023-11-28T11:11:22.988972+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:53.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990325+0000","last_change":"2023-11-28T11:11:22.990325+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990325+0000","last_undegraded":"2023-11-28T11:11:22.990325+0000","last_fullsized":"2023-11-28T11:11:22.990325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:00.027787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993387+0000","last_change":"2023-11-28T11:11:22.993387+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993387+0000","last_undegraded":"2023-11-28T11:11:22.993387+0000","last_fullsized":"2023-11-28T11:11:22.993387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.992405+0000","last_change":"2023-11-28T11:11:22.992405+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.992405+0000","last_undegraded":"2023-11-28T11:11:22.992405+0000","last_fullsized":"2023-11-28T11:11:22.992405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:06.471882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987642+0000","last_change":"2023-11-28T11:11:22.987642+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987642+0000","last_undegraded":"2023-11-28T11:11:22.987642+0000","last_fullsized":"2023-11-28T11:11:22.987642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:03.658510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987726+0000","last_change":"2023-11-28T11:11:22.987726+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987726+0000","last_undegraded":"2023-11-28T11:11:22.987726+0000","last_fullsized":"2023-11-28T11:11:22.987726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994084+0000","last_change":"2023-11-28T11:11:22.994084+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994084+0000","last_undegraded":"2023-11-28T11:11:22.994084+0000","last_fullsized":"2023-11-28T11:11:22.994084+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:34.752567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987109+0000","last_change":"2023-11-28T11:11:22.987109+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987109+0000","last_undegraded":"2023-11-28T11:11:22.987109+0000","last_fullsized":"2023-11-28T11:11:22.987109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:21.311957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990182+0000","last_change":"2023-11-28T11:11:22.990182+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990182+0000","last_undegraded":"2023-11-28T11:11:22.990182+0000","last_fullsized":"2023-11-28T11:11:22.990182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:03.523669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993780+0000","last_change":"2023-11-28T11:11:22.993780+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993780+0000","last_undegraded":"2023-11-28T11:11:22.993780+0000","last_fullsized":"2023-11-28T11:11:22.993780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:20.035165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.986745+0000","last_change":"2023-11-28T11:11:22.986745+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.986745+0000","last_undegraded":"2023-11-28T11:11:22.986745+0000","last_fullsized":"2023-11-28T11:11:22.986745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:28.037670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989275+0000","last_change":"2023-11-28T11:11:22.989275+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989275+0000","last_undegraded":"2023-11-28T11:11:22.989275+0000","last_fullsized":"2023-11-28T11:11:22.989275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:43:12.693701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989274+0000","last_change":"2023-11-28T11:11:22.989274+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989274+0000","last_undegraded":"2023-11-28T11:11:22.989274+0000","last_fullsized":"2023-11-28T11:11:22.989274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:10.349250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.988966+0000","last_change":"2023-11-28T11:11:22.988966+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.988966+0000","last_undegraded":"2023-11-28T11:11:22.988966+0000","last_fullsized":"2023-11-28T11:11:22.988966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:36.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987636+0000","last_change":"2023-11-28T11:11:22.987636+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987636+0000","last_undegraded":"2023-11-28T11:11:22.987636+0000","last_fullsized":"2023-11-28T11:11:22.987636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990676+0000","last_change":"2023-11-28T11:11:22.990676+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990676+0000","last_undegraded":"2023-11-28T11:11:22.990676+0000","last_fullsized":"2023-11-28T11:11:22.990676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:26.554018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993143+0000","last_change":"2023-11-28T11:11:22.993143+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993143+0000","last_undegraded":"2023-11-28T11:11:22.993143+0000","last_fullsized":"2023-11-28T11:11:22.993143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:27.874016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990264+0000","last_change":"2023-11-28T11:11:22.990264+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990264+0000","last_undegraded":"2023-11-28T11:11:22.990264+0000","last_fullsized":"2023-11-28T11:11:22.990264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:09.661474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:20.094641+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:11:20.094641+0000","last_peered":"2023-11-28T11:11:20.094641+0000","last_clean":"2023-11-28T11:11:20.094641+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:11:20.094641+0000","last_undegraded":"2023-11-28T11:11:20.094641+0000","last_fullsized":"2023-11-28T11:11:20.094641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994667+0000","last_change":"2023-11-28T11:11:22.994667+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994667+0000","last_undegraded":"2023-11-28T11:11:22.994667+0000","last_fullsized":"2023-11-28T11:11:22.994667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.118149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.986707+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:11:21.986707+0000","last_peered":"2023-11-28T11:11:21.986707+0000","last_clean":"2023-11-28T11:11:21.986707+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:11:21.986707+0000","last_undegraded":"2023-11-28T11:11:21.986707+0000","last_fullsized":"2023-11-28T11:11:21.986707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989562+0000","last_change":"2023-11-28T11:11:22.989562+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989562+0000","last_undegraded":"2023-11-28T11:11:22.989562+0000","last_fullsized":"2023-11-28T11:11:22.989562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:42.594460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.399337+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:11:21.399337+0000","last_peered":"2023-11-28T11:11:21.399337+0000","last_clean":"2023-11-28T11:11:21.399337+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:11:21.399337+0000","last_undegraded":"2023-11-28T11:11:21.399337+0000","last_fullsized":"2023-11-28T11:11:21.399337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":194,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987549+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:11:21.987549+0000","last_peered":"2023-11-28T11:11:21.987549+0000","last_clean":"2023-11-28T11:11:21.987549+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:11:21.987549+0000","last_undegraded":"2023-11-28T11:11:21.987549+0000","last_fullsized":"2023-11-28T11:11:21.987549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.986627+0000","last_change":"2023-11-28T11:11:22.986627+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.986627+0000","last_undegraded":"2023-11-28T11:11:22.986627+0000","last_fullsized":"2023-11-28T11:11:22.986627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:01.938679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987547+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:11:21.987547+0000","last_peered":"2023-11-28T11:11:21.987547+0000","last_clean":"2023-11-28T11:11:21.987547+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:11:21.987547+0000","last_undegraded":"2023-11-28T11:11:21.987547+0000","last_fullsized":"2023-11-28T11:11:21.987547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993422+0000","last_change":"2023-11-28T11:11:22.993422+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993422+0000","last_undegraded":"2023-11-28T11:11:22.993422+0000","last_fullsized":"2023-11-28T11:11:22.993422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:25.947439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987514+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:11:21.987514+0000","last_peered":"2023-11-28T11:11:21.987514+0000","last_clean":"2023-11-28T11:11:21.987514+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:11:21.987514+0000","last_undegraded":"2023-11-28T11:11:21.987514+0000","last_fullsized":"2023-11-28T11:11:21.987514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993156+0000","last_change":"2023-11-28T11:11:22.993156+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993156+0000","last_undegraded":"2023-11-28T11:11:22.993156+0000","last_fullsized":"2023-11-28T11:11:22.993156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:20.094641+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:11:20.094641+0000","last_peered":"2023-11-28T11:11:20.094641+0000","last_clean":"2023-11-28T11:11:20.094641+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:11:20.094641+0000","last_undegraded":"2023-11-28T11:11:20.094641+0000","last_fullsized":"2023-11-28T11:11:20.094641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993454+0000","last_change":"2023-11-28T11:11:22.993454+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993454+0000","last_undegraded":"2023-11-28T11:11:22.993454+0000","last_fullsized":"2023-11-28T11:11:22.993454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987205+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:11:21.987205+0000","last_peered":"2023-11-28T11:11:21.987205+0000","last_clean":"2023-11-28T11:11:21.987205+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:11:21.987205+0000","last_undegraded":"2023-11-28T11:11:21.987205+0000","last_fullsized":"2023-11-28T11:11:21.987205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.991826+0000","last_change":"2023-11-28T11:11:22.991826+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.991826+0000","last_undegraded":"2023-11-28T11:11:22.991826+0000","last_fullsized":"2023-11-28T11:11:22.991826+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:44.971764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987232+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:11:21.987232+0000","last_peered":"2023-11-28T11:11:21.987232+0000","last_clean":"2023-11-28T11:11:21.987232+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:11:21.987232+0000","last_undegraded":"2023-11-28T11:11:21.987232+0000","last_fullsized":"2023-11-28T11:11:21.987232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10576,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361264,"statfs":{"total":96636764160,"available":96625934336,"internally_reserved":0,"allocated":1318912,"data_stored":925778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.376},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":708608,"data_stored":327559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":1.544,"15min":1.544},"last":0.59699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.337,"15min":1.337},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":1.665,"15min":1.665},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71399999999999997,"5min":1.762,"15min":1.762},"last":0.69899999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.60599999999999998,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72899999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71799999999999997,"5min":1.002,"15min":1.002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70399999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.877},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10512,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361328,"statfs":{"total":96636764160,"available":96625999872,"internally_reserved":0,"allocated":1318912,"data_stored":925778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38,"15min":0.39000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58699999999999997,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.504,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34499999999999997,"15min":0.34899999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.57399999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40999999999999998,"15min":0.41899999999999998},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.48499999999999999,"5min":1.363,"15min":1.363},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73799999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.752,"5min":1.54,"15min":1.54},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":1.48,"15min":1.48},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72899999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.755},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":1.004,"15min":1.004},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9660,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362180,"statfs":{"total":96636764160,"available":96626872320,"internally_reserved":0,"allocated":708608,"data_stored":327559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47999999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32500000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.378}]},{"osd":3,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40100000000000002},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.248},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58899999999999997,"5min":0.871,"15min":0.871},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72999999999999998,"5min":0.872,"15min":0.872},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.47399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9356,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362484,"statfs":{"total":96636764160,"available":96627183616,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.27000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.377},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.33400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78000000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75800000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80900000000000005,"5min":1.024,"15min":1.024},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83499999999999996,"5min":1.05,"15min":1.05},"last":0.625}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9612,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362228,"statfs":{"total":96636764160,"available":96626921472,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.499,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35799999999999998,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.219}]},{"osd":1,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":1.054,"15min":1.054},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53400000000000003,"5min":1.147,"15min":1.147},"last":0.251}]},{"osd":2,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.58699999999999997,"15min":0.622},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":5.133,"15min":5.133},"last":0.28100000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68100000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72599999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.194,"15min":1.194},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":1.077,"15min":1.077},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9612,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362228,"statfs":{"total":96636764160,"available":96626921472,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67400000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.246},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.27600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.29399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.747,"15min":0.747},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72499999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.30099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9420,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362420,"statfs":{"total":96636764160,"available":96627118080,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.81299999999999994,"15min":0.86099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84799999999999998,"15min":0.90000000000000002},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70999999999999996,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":6.601,"15min":6.601},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.375}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:28.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.978+0000 7f404337c700 1 Processor -- start 2023-11-28T11:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.978+0000 7f404337c700 1 -- start start 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f404337c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f403c15d0e0 0x7f403c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f404337c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f403c0b1fb0 0x7f403c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f404337c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0cbfb0 con 0x7f403c15cd10 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f404337c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c06cee0 con 0x7f403c15d0e0 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f404337c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c0cb710 con 0x7f403c0b1fb0 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f403c15d0e0 0x7f403c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f403c15d0e0 0x7f403c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52748/0 (socket says 172.21.15.83:52748) 2023-11-28T11:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 -- 172.21.15.83:0/3810447851 learned_addr learned my addr 172.21.15.83:0/3810447851 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4041919700 1 --2- 172.21.15.83:0/3810447851 >> v2:172.21.15.90:3300/0 conn(0x7f403c0b1fb0 0x7f403c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 -- 172.21.15.83:0/3810447851 >> v2:172.21.15.90:3300/0 conn(0x7f403c0b1fb0 msgr2=0x7f403c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 --2- 172.21.15.83:0/3810447851 >> v2:172.21.15.90:3300/0 conn(0x7f403c0b1fb0 0x7f403c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 -- 172.21.15.83:0/3810447851 >> v1:172.21.15.90:6789/0 conn(0x7f403c15cd10 legacy=0x7f403c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.979+0000 7f4040917700 1 -- 172.21.15.83:0/3810447851 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0cb4e0 con 0x7f403c15d0e0 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f4040917700 1 --2- 172.21.15.83:0/3810447851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c15d0e0 0x7f403c0ca970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f403c0cc790 tx=0x7f4038009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8461b320c2e9bc3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f4033fff700 1 -- 172.21.15.83:0/3810447851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038004230 con 0x7f403c15d0e0 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f4033fff700 1 -- 172.21.15.83:0/3810447851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038004390 con 0x7f403c15d0e0 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f4033fff700 1 -- 172.21.15.83:0/3810447851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380036a0 con 0x7f403c15d0e0 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 -- 172.21.15.83:0/3810447851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c15d0e0 msgr2=0x7f403c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 --2- 172.21.15.83:0/3810447851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c15d0e0 0x7f403c0ca970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f403c0cc790 tx=0x7f4038009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 -- 172.21.15.83:0/3810447851 shutdown_connections 2023-11-28T11:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 --2- 172.21.15.83:0/3810447851 >> v2:172.21.15.90:3300/0 conn(0x7f403c0b1fb0 0x7f403c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 --2- 172.21.15.83:0/3810447851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c15d0e0 0x7f403c0ca970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 -- 172.21.15.83:0/3810447851 >> 172.21.15.83:0/3810447851 conn(0x7f403c06e9a0 msgr2=0x7f403c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 -- 172.21.15.83:0/3810447851 shutdown_connections 2023-11-28T11:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.980+0000 7f404337c700 1 -- 172.21.15.83:0/3810447851 wait complete. 2023-11-28T11:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.982+0000 7f404337c700 1 Processor -- start 2023-11-28T11:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.982+0000 7f404337c700 1 -- start start 2023-11-28T11:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f404337c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 0x7f403c15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f404337c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f403c15cd10 0x7f403c0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f404337c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f403c15d0e0 0x7f403c0b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f404337c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0b2df0 con 0x7f403c0b1fb0 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f404337c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0b30d0 con 0x7f403c15d0e0 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f404337c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0b33b0 con 0x7f403c15cd10 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 0x7f403c15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 0x7f403c15a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52752/0 (socket says 172.21.15.83:52752) 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 -- 172.21.15.83:0/1908894892 learned_addr learned my addr 172.21.15.83:0/1908894892 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4040917700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f403c15cd10 0x7f403c0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041919700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f403c15d0e0 0x7f403c0b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 -- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f403c15cd10 msgr2=0x7f403c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f403c15cd10 0x7f403c0adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 -- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f403c15d0e0 msgr2=0x7f403c0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f403c15d0e0 0x7f403c0b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 -- 172.21.15.83:0/1908894892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009550 con 0x7f403c0b1fb0 2023-11-28T11:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041919700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f403c15d0e0 0x7f403c0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.983+0000 7f4041118700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 0x7f403c15a340 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c4a0 tx=0x7f402c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.984+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0054e0 con 0x7f403c0b1fb0 2023-11-28T11:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.984+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c003680 con 0x7f403c0b1fb0 2023-11-28T11:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.984+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c0cbcd0 con 0x7f403c0b1fb0 2023-11-28T11:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.984+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c003a30 con 0x7f403c0b1fb0 2023-11-28T11:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.984+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c0cb9f0 con 0x7f403c0b1fb0 2023-11-28T11:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.985+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c06b9b0 con 0x7f403c0b1fb0 2023-11-28T11:11:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.985+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f402c019030 con 0x7f403c0b1fb0 2023-11-28T11:11:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.985+0000 7f4031ffb700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40200682f0 0x7f402006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.985+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f402c085150 con 0x7f403c0b1fb0 2023-11-28T11:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.986+0000 7f4040917700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40200682f0 0x7f402006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.986+0000 7f4040917700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40200682f0 0x7f402006a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4038009130 tx=0x7f4038008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:28.990+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f402c051870 con 0x7f403c0b1fb0 2023-11-28T11:11:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.170+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f403c074320 con 0x7f40200682f0 2023-11-28T11:11:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.178+0000 7f4031ffb700 1 -- 172.21.15.83:0/1908894892 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300968 (secure 0 0 0) 0x7f403c074320 con 0x7f40200682f0 2023-11-28T11:11:29.184 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40200682f0 msgr2=0x7f402006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40200682f0 0x7f402006a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4038009130 tx=0x7f4038008030 comp rx=0 tx=0).stop 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 msgr2=0x7f403c15a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 0x7f403c15a340 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c4a0 tx=0x7f402c00c7b0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 shutdown_connections 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f403c15d0e0 0x7f403c0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40200682f0 0x7f402006a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f403c0b1fb0 0x7f403c15a340 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 --2- 172.21.15.83:0/1908894892 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f403c15cd10 0x7f403c0adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 >> 172.21.15.83:0/1908894892 conn(0x7f403c06e9a0 msgr2=0x7f403c0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:29.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 shutdown_connections 2023-11-28T11:11:29.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.182+0000 7f404337c700 1 -- 172.21.15.83:0/1908894892 wait complete. 2023-11-28T11:11:29.188 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:11:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-28T11:11:27.605173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78024,"kb_used_data":6792,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":754896696,"statfs":{"total":773094113280,"available":773014216704,"internally_reserved":0,"allocated":6955008,"data_stored":3848314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619076"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.991933+0000","last_change":"2023-11-28T11:11:22.991933+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.991933+0000","last_undegraded":"2023-11-28T11:11:22.991933+0000","last_fullsized":"2023-11-28T11:11:22.991933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:51.009482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989322+0000","last_change":"2023-11-28T11:11:22.989322+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989322+0000","last_undegraded":"2023-11-28T11:11:22.989322+0000","last_fullsized":"2023-11-28T11:11:22.989322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:03.348424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990291+0000","last_change":"2023-11-28T11:11:22.990291+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990291+0000","last_undegraded":"2023-11-28T11:11:22.990291+0000","last_fullsized":"2023-11-28T11:11:22.990291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:12.037178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993692+0000","last_change":"2023-11-28T11:11:22.993692+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993692+0000","last_undegraded":"2023-11-28T11:11:22.993692+0000","last_fullsized":"2023-11-28T11:11:22.993692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:40.682923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.992400+0000","last_change":"2023-11-28T11:11:22.992400+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.992400+0000","last_undegraded":"2023-11-28T11:11:22.992400+0000","last_fullsized":"2023-11-28T11:11:22.992400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:08.956676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994804+0000","last_change":"2023-11-28T11:11:22.994804+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994804+0000","last_undegraded":"2023-11-28T11:11:22.994804+0000","last_fullsized":"2023-11-28T11:11:22.994804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:49.819742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987647+0000","last_change":"2023-11-28T11:11:22.987647+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987647+0000","last_undegraded":"2023-11-28T11:11:22.987647+0000","last_fullsized":"2023-11-28T11:11:22.987647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.988972+0000","last_change":"2023-11-28T11:11:22.988972+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.988972+0000","last_undegraded":"2023-11-28T11:11:22.988972+0000","last_fullsized":"2023-11-28T11:11:22.988972+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:53.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990325+0000","last_change":"2023-11-28T11:11:22.990325+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990325+0000","last_undegraded":"2023-11-28T11:11:22.990325+0000","last_fullsized":"2023-11-28T11:11:22.990325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:00.027787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993387+0000","last_change":"2023-11-28T11:11:22.993387+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993387+0000","last_undegraded":"2023-11-28T11:11:22.993387+0000","last_fullsized":"2023-11-28T11:11:22.993387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.992405+0000","last_change":"2023-11-28T11:11:22.992405+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.992405+0000","last_undegraded":"2023-11-28T11:11:22.992405+0000","last_fullsized":"2023-11-28T11:11:22.992405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:06.471882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987642+0000","last_change":"2023-11-28T11:11:22.987642+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987642+0000","last_undegraded":"2023-11-28T11:11:22.987642+0000","last_fullsized":"2023-11-28T11:11:22.987642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:03.658510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987726+0000","last_change":"2023-11-28T11:11:22.987726+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987726+0000","last_undegraded":"2023-11-28T11:11:22.987726+0000","last_fullsized":"2023-11-28T11:11:22.987726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994084+0000","last_change":"2023-11-28T11:11:22.994084+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994084+0000","last_undegraded":"2023-11-28T11:11:22.994084+0000","last_fullsized":"2023-11-28T11:11:22.994084+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:34.752567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987109+0000","last_change":"2023-11-28T11:11:22.987109+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987109+0000","last_undegraded":"2023-11-28T11:11:22.987109+0000","last_fullsized":"2023-11-28T11:11:22.987109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:21.311957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990182+0000","last_change":"2023-11-28T11:11:22.990182+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990182+0000","last_undegraded":"2023-11-28T11:11:22.990182+0000","last_fullsized":"2023-11-28T11:11:22.990182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:03.523669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993780+0000","last_change":"2023-11-28T11:11:22.993780+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993780+0000","last_undegraded":"2023-11-28T11:11:22.993780+0000","last_fullsized":"2023-11-28T11:11:22.993780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:20.035165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.986745+0000","last_change":"2023-11-28T11:11:22.986745+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.986745+0000","last_undegraded":"2023-11-28T11:11:22.986745+0000","last_fullsized":"2023-11-28T11:11:22.986745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:28.037670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989275+0000","last_change":"2023-11-28T11:11:22.989275+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989275+0000","last_undegraded":"2023-11-28T11:11:22.989275+0000","last_fullsized":"2023-11-28T11:11:22.989275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:43:12.693701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989274+0000","last_change":"2023-11-28T11:11:22.989274+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989274+0000","last_undegraded":"2023-11-28T11:11:22.989274+0000","last_fullsized":"2023-11-28T11:11:22.989274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:10.349250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.988966+0000","last_change":"2023-11-28T11:11:22.988966+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.988966+0000","last_undegraded":"2023-11-28T11:11:22.988966+0000","last_fullsized":"2023-11-28T11:11:22.988966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:36.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.987636+0000","last_change":"2023-11-28T11:11:22.987636+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.987636+0000","last_undegraded":"2023-11-28T11:11:22.987636+0000","last_fullsized":"2023-11-28T11:11:22.987636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990676+0000","last_change":"2023-11-28T11:11:22.990676+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990676+0000","last_undegraded":"2023-11-28T11:11:22.990676+0000","last_fullsized":"2023-11-28T11:11:22.990676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:26.554018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993143+0000","last_change":"2023-11-28T11:11:22.993143+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993143+0000","last_undegraded":"2023-11-28T11:11:22.993143+0000","last_fullsized":"2023-11-28T11:11:22.993143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:27.874016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.990264+0000","last_change":"2023-11-28T11:11:22.990264+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.990264+0000","last_undegraded":"2023-11-28T11:11:22.990264+0000","last_fullsized":"2023-11-28T11:11:22.990264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:09.661474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:20.094641+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:11:20.094641+0000","last_peered":"2023-11-28T11:11:20.094641+0000","last_clean":"2023-11-28T11:11:20.094641+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:11:20.094641+0000","last_undegraded":"2023-11-28T11:11:20.094641+0000","last_fullsized":"2023-11-28T11:11:20.094641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.994667+0000","last_change":"2023-11-28T11:11:22.994667+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.994667+0000","last_undegraded":"2023-11-28T11:11:22.994667+0000","last_fullsized":"2023-11-28T11:11:22.994667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.118149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.986707+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:11:21.986707+0000","last_peered":"2023-11-28T11:11:21.986707+0000","last_clean":"2023-11-28T11:11:21.986707+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:11:21.986707+0000","last_undegraded":"2023-11-28T11:11:21.986707+0000","last_fullsized":"2023-11-28T11:11:21.986707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.989562+0000","last_change":"2023-11-28T11:11:22.989562+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.989562+0000","last_undegraded":"2023-11-28T11:11:22.989562+0000","last_fullsized":"2023-11-28T11:11:22.989562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:42.594460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.399337+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:11:21.399337+0000","last_peered":"2023-11-28T11:11:21.399337+0000","last_clean":"2023-11-28T11:11:21.399337+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:11:21.399337+0000","last_undegraded":"2023-11-28T11:11:21.399337+0000","last_fullsized":"2023-11-28T11:11:21.399337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":194,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987549+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:11:21.987549+0000","last_peered":"2023-11-28T11:11:21.987549+0000","last_clean":"2023-11-28T11:11:21.987549+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:11:21.987549+0000","last_undegraded":"2023-11-28T11:11:21.987549+0000","last_fullsized":"2023-11-28T11:11:21.987549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.986627+0000","last_change":"2023-11-28T11:11:22.986627+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.986627+0000","last_undegraded":"2023-11-28T11:11:22.986627+0000","last_fullsized":"2023-11-28T11:11:22.986627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:01.938679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987547+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:11:21.987547+0000","last_peered":"2023-11-28T11:11:21.987547+0000","last_clean":"2023-11-28T11:11:21.987547+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:11:21.987547+0000","last_undegraded":"2023-11-28T11:11:21.987547+0000","last_fullsized":"2023-11-28T11:11:21.987547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993422+0000","last_change":"2023-11-28T11:11:22.993422+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993422+0000","last_undegraded":"2023-11-28T11:11:22.993422+0000","last_fullsized":"2023-11-28T11:11:22.993422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:25.947439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987514+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:11:21.987514+0000","last_peered":"2023-11-28T11:11:21.987514+0000","last_clean":"2023-11-28T11:11:21.987514+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:11:21.987514+0000","last_undegraded":"2023-11-28T11:11:21.987514+0000","last_fullsized":"2023-11-28T11:11:21.987514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993156+0000","last_change":"2023-11-28T11:11:22.993156+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993156+0000","last_undegraded":"2023-11-28T11:11:22.993156+0000","last_fullsized":"2023-11-28T11:11:22.993156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:20.094641+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:11:20.094641+0000","last_peered":"2023-11-28T11:11:20.094641+0000","last_clean":"2023-11-28T11:11:20.094641+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:11:20.094641+0000","last_undegraded":"2023-11-28T11:11:20.094641+0000","last_fullsized":"2023-11-28T11:11:20.094641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.993454+0000","last_change":"2023-11-28T11:11:22.993454+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.993454+0000","last_undegraded":"2023-11-28T11:11:22.993454+0000","last_fullsized":"2023-11-28T11:11:22.993454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987205+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:11:21.987205+0000","last_peered":"2023-11-28T11:11:21.987205+0000","last_clean":"2023-11-28T11:11:21.987205+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:11:21.987205+0000","last_undegraded":"2023-11-28T11:11:21.987205+0000","last_fullsized":"2023-11-28T11:11:21.987205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:11:23.983190+0000","last_change":"2023-11-28T11:11:23.983190+0000","last_active":"2023-11-28T11:11:23.983190+0000","last_peered":"2023-11-28T11:11:23.983190+0000","last_clean":"2023-11-28T11:11:23.983190+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:23.983190+0000","last_undegraded":"2023-11-28T11:11:23.983190+0000","last_fullsized":"2023-11-28T11:11:23.983190+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:11:22.991826+0000","last_change":"2023-11-28T11:11:22.991826+0000","last_active":"2023-11-28T11:11:22.979612+0000","last_peered":"2023-11-28T11:11:22.979612+0000","last_clean":"2023-11-28T11:11:22.979612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:11:22.991826+0000","last_undegraded":"2023-11-28T11:11:22.991826+0000","last_fullsized":"2023-11-28T11:11:22.991826+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:44.971764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:11:21.987232+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:11:21.987232+0000","last_peered":"2023-11-28T11:11:21.987232+0000","last_clean":"2023-11-28T11:11:21.987232+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:11:21.987232+0000","last_undegraded":"2023-11-28T11:11:21.987232+0000","last_fullsized":"2023-11-28T11:11:21.987232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10576,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361264,"statfs":{"total":96636764160,"available":96625934336,"internally_reserved":0,"allocated":1318912,"data_stored":925778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.376},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":708608,"data_stored":327559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":1.544,"15min":1.544},"last":0.59699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.337,"15min":1.337},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":1.665,"15min":1.665},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71399999999999997,"5min":1.762,"15min":1.762},"last":0.69899999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.60599999999999998,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72899999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71799999999999997,"5min":1.002,"15min":1.002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70399999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.877},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10512,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361328,"statfs":{"total":96636764160,"available":96625999872,"internally_reserved":0,"allocated":1318912,"data_stored":925778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38,"15min":0.39000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58699999999999997,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.504,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34499999999999997,"15min":0.34899999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.57399999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40999999999999998,"15min":0.41899999999999998},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.48499999999999999,"5min":1.363,"15min":1.363},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73799999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.752,"5min":1.54,"15min":1.54},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":1.48,"15min":1.48},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72899999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.755},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":1.004,"15min":1.004},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9660,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362180,"statfs":{"total":96636764160,"available":96626872320,"internally_reserved":0,"allocated":708608,"data_stored":327559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47999999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32500000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.378}]},{"osd":3,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40100000000000002},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.248},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58899999999999997,"5min":0.871,"15min":0.871},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72999999999999998,"5min":0.872,"15min":0.872},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.47399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9356,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362484,"statfs":{"total":96636764160,"available":96627183616,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.27000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.377},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.33400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78000000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75800000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80900000000000005,"5min":1.024,"15min":1.024},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83499999999999996,"5min":1.05,"15min":1.05},"last":0.625}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9612,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362228,"statfs":{"total":96636764160,"available":96626921472,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.499,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35799999999999998,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.219}]},{"osd":1,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":1.054,"15min":1.054},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53400000000000003,"5min":1.147,"15min":1.147},"last":0.251}]},{"osd":2,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.58699999999999997,"15min":0.622},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":5.133,"15min":5.133},"last":0.28100000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68100000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72599999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.194,"15min":1.194},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":1.077,"15min":1.077},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9612,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362228,"statfs":{"total":96636764160,"available":96626921472,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67400000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.246},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.27600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.29399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.747,"15min":0.747},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72499999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.30099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9420,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362420,"statfs":{"total":96636764160,"available":96627118080,"internally_reserved":0,"allocated":724992,"data_stored":335410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.81299999999999994,"15min":0.86099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84799999999999998,"15min":0.90000000000000002},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70999999999999996,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":6.601,"15min":6.601},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.375}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:29.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-11-28T11:11:29.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-11-28T11:11:29.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-11-28T11:11:29.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-11-28T11:11:29.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-11-28T11:11:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-11-28T11:11:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.991933+0000', 'last_change': '2023-11-28T11:11:22.991933+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.991933+0000', 'last_undegraded': '2023-11-28T11:11:22.991933+0000', 'last_fullsized': '2023-11-28T11:11:22.991933+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:01:51.009482+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.989322+0000', 'last_change': '2023-11-28T11:11:22.989322+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.989322+0000', 'last_undegraded': '2023-11-28T11:11:22.989322+0000', 'last_fullsized': '2023-11-28T11:11:22.989322+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:49:03.348424+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-28T11:11:29.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.990291+0000', 'last_change': '2023-11-28T11:11:22.990291+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.990291+0000', 'last_undegraded': '2023-11-28T11:11:22.990291+0000', 'last_fullsized': '2023-11-28T11:11:22.990291+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:37:12.037178+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993692+0000', 'last_change': '2023-11-28T11:11:22.993692+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993692+0000', 'last_undegraded': '2023-11-28T11:11:22.993692+0000', 'last_fullsized': '2023-11-28T11:11:22.993692+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:56:40.682923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-28T11:11:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.992400+0000', 'last_change': '2023-11-28T11:11:22.992400+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.992400+0000', 'last_undegraded': '2023-11-28T11:11:22.992400+0000', 'last_fullsized': '2023-11-28T11:11:22.992400+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:54:08.956676+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.994804+0000', 'last_change': '2023-11-28T11:11:22.994804+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.994804+0000', 'last_undegraded': '2023-11-28T11:11:22.994804+0000', 'last_fullsized': '2023-11-28T11:11:22.994804+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:59:49.819742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-28T11:11:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.987647+0000', 'last_change': '2023-11-28T11:11:22.987647+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.987647+0000', 'last_undegraded': '2023-11-28T11:11:22.987647+0000', 'last_fullsized': '2023-11-28T11:11:22.987647+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.988972+0000', 'last_change': '2023-11-28T11:11:22.988972+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.988972+0000', 'last_undegraded': '2023-11-28T11:11:22.988972+0000', 'last_fullsized': '2023-11-28T11:11:22.988972+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:10:53.282086+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.990325+0000', 'last_change': '2023-11-28T11:11:22.990325+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.990325+0000', 'last_undegraded': '2023-11-28T11:11:22.990325+0000', 'last_fullsized': '2023-11-28T11:11:22.990325+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:32:00.027787+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-28T11:11:29.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993387+0000', 'last_change': '2023-11-28T11:11:22.993387+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993387+0000', 'last_undegraded': '2023-11-28T11:11:22.993387+0000', 'last_fullsized': '2023-11-28T11:11:22.993387+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.992405+0000', 'last_change': '2023-11-28T11:11:22.992405+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.992405+0000', 'last_undegraded': '2023-11-28T11:11:22.992405+0000', 'last_fullsized': '2023-11-28T11:11:22.992405+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:59:06.471882+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-28T11:11:29.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.987642+0000', 'last_change': '2023-11-28T11:11:22.987642+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.987642+0000', 'last_undegraded': '2023-11-28T11:11:22.987642+0000', 'last_fullsized': '2023-11-28T11:11:22.987642+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:39:03.658510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.987726+0000', 'last_change': '2023-11-28T11:11:22.987726+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.987726+0000', 'last_undegraded': '2023-11-28T11:11:22.987726+0000', 'last_fullsized': '2023-11-28T11:11:22.987726+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-28T11:11:29.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.994084+0000', 'last_change': '2023-11-28T11:11:22.994084+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.994084+0000', 'last_undegraded': '2023-11-28T11:11:22.994084+0000', 'last_fullsized': '2023-11-28T11:11:22.994084+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:49:34.752567+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.987109+0000', 'last_change': '2023-11-28T11:11:22.987109+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.987109+0000', 'last_undegraded': '2023-11-28T11:11:22.987109+0000', 'last_fullsized': '2023-11-28T11:11:22.987109+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:01:21.311957+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.990182+0000', 'last_change': '2023-11-28T11:11:22.990182+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.990182+0000', 'last_undegraded': '2023-11-28T11:11:22.990182+0000', 'last_fullsized': '2023-11-28T11:11:22.990182+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T11:12:03.523669+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:11:29.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993780+0000', 'last_change': '2023-11-28T11:11:22.993780+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993780+0000', 'last_undegraded': '2023-11-28T11:11:22.993780+0000', 'last_fullsized': '2023-11-28T11:11:22.993780+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:53:20.035165+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.986745+0000', 'last_change': '2023-11-28T11:11:22.986745+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.986745+0000', 'last_undegraded': '2023-11-28T11:11:22.986745+0000', 'last_fullsized': '2023-11-28T11:11:22.986745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:26:28.037670+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.989275+0000', 'last_change': '2023-11-28T11:11:22.989275+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.989275+0000', 'last_undegraded': '2023-11-28T11:11:22.989275+0000', 'last_fullsized': '2023-11-28T11:11:22.989275+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:43:12.693701+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.989274+0000', 'last_change': '2023-11-28T11:11:22.989274+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.989274+0000', 'last_undegraded': '2023-11-28T11:11:22.989274+0000', 'last_fullsized': '2023-11-28T11:11:22.989274+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:19:10.349250+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-11-28T11:11:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.988966+0000', 'last_change': '2023-11-28T11:11:22.988966+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.988966+0000', 'last_undegraded': '2023-11-28T11:11:22.988966+0000', 'last_fullsized': '2023-11-28T11:11:22.988966+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:01:36.973117+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:11:29.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-28T11:11:29.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-28T11:11:29.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.987636+0000', 'last_change': '2023-11-28T11:11:22.987636+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.987636+0000', 'last_undegraded': '2023-11-28T11:11:22.987636+0000', 'last_fullsized': '2023-11-28T11:11:22.987636+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.990676+0000', 'last_change': '2023-11-28T11:11:22.990676+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.990676+0000', 'last_undegraded': '2023-11-28T11:11:22.990676+0000', 'last_fullsized': '2023-11-28T11:11:22.990676+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:00:26.554018+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-28T11:11:29.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993143+0000', 'last_change': '2023-11-28T11:11:22.993143+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993143+0000', 'last_undegraded': '2023-11-28T11:11:22.993143+0000', 'last_fullsized': '2023-11-28T11:11:22.993143+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:39:27.874016+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.990264+0000', 'last_change': '2023-11-28T11:11:22.990264+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.990264+0000', 'last_undegraded': '2023-11-28T11:11:22.990264+0000', 'last_fullsized': '2023-11-28T11:11:22.990264+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:22:09.661474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.994667+0000', 'last_change': '2023-11-28T11:11:22.994667+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.994667+0000', 'last_undegraded': '2023-11-28T11:11:22.994667+0000', 'last_fullsized': '2023-11-28T11:11:22.994667+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:14:57.118149+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-28T11:11:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.989562+0000', 'last_change': '2023-11-28T11:11:22.989562+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.989562+0000', 'last_undegraded': '2023-11-28T11:11:22.989562+0000', 'last_fullsized': '2023-11-28T11:11:22.989562+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:13:42.594460+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.986627+0000', 'last_change': '2023-11-28T11:11:22.986627+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.986627+0000', 'last_undegraded': '2023-11-28T11:11:22.986627+0000', 'last_fullsized': '2023-11-28T11:11:22.986627+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:41:01.938679+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-28T11:11:29.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993422+0000', 'last_change': '2023-11-28T11:11:22.993422+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993422+0000', 'last_undegraded': '2023-11-28T11:11:22.993422+0000', 'last_fullsized': '2023-11-28T11:11:22.993422+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:18:25.947439+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993156+0000', 'last_change': '2023-11-28T11:11:22.993156+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993156+0000', 'last_undegraded': '2023-11-28T11:11:22.993156+0000', 'last_fullsized': '2023-11-28T11:11:22.993156+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.993454+0000', 'last_change': '2023-11-28T11:11:22.993454+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.993454+0000', 'last_undegraded': '2023-11-28T11:11:22.993454+0000', 'last_fullsized': '2023-11-28T11:11:22.993454+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-28T11:11:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:11:23.983190+0000', 'last_change': '2023-11-28T11:11:23.983190+0000', 'last_active': '2023-11-28T11:11:23.983190+0000', 'last_peered': '2023-11-28T11:11:23.983190+0000', 'last_clean': '2023-11-28T11:11:23.983190+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:23.983190+0000', 'last_undegraded': '2023-11-28T11:11:23.983190+0000', 'last_fullsized': '2023-11-28T11:11:23.983190+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:23.983190+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:29.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-28T11:11:29.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:11:22.991826+0000', 'last_change': '2023-11-28T11:11:22.991826+0000', 'last_active': '2023-11-28T11:11:22.979612+0000', 'last_peered': '2023-11-28T11:11:22.979612+0000', 'last_clean': '2023-11-28T11:11:22.979612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:11:22.991826+0000', 'last_undegraded': '2023-11-28T11:11:22.991826+0000', 'last_fullsized': '2023-11-28T11:11:22.991826+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'last_clean_scrub_stamp': '2023-11-28T11:11:22.979612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:57:44.971764+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:11:29.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T11:11:29.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.352+0000 7f166ae5c700 1 Processor -- start 2023-11-28T11:11:29.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.352+0000 7f166ae5c700 1 -- start start 2023-11-28T11:11:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f166ae5c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16640b88d0 0x7f16640cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f166ae5c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f16640b5170 0x7f16640d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f166ae5c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16640d0c50 con 0x7f1664160e60 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f166ae5c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16640b8ca0 con 0x7f16640b5170 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f166ae5c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16640cd760 con 0x7f16640b88d0 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f16693f9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f16640b5170 0x7f16640d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f1663fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1664160e60 0x7f16640b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50624/0 (socket says 172.21.15.83:50624) 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.353+0000 7f1663fff700 1 -- 172.21.15.83:0/1425045985 learned_addr learned my addr 172.21.15.83:0/1425045985 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f1668bf8700 1 --2- 172.21.15.83:0/1425045985 >> v2:172.21.15.90:3300/0 conn(0x7f16640b88d0 0x7f16640cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16693f9700 1 -- 172.21.15.83:0/1425045985 >> v2:172.21.15.90:3300/0 conn(0x7f16640b88d0 msgr2=0x7f16640cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16693f9700 1 --2- 172.21.15.83:0/1425045985 >> v2:172.21.15.90:3300/0 conn(0x7f16640b88d0 0x7f16640cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16693f9700 1 -- 172.21.15.83:0/1425045985 >> v1:172.21.15.83:6789/0 conn(0x7f1664160e60 legacy=0x7f16640b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16637fe700 1 -- 172.21.15.83:0/1425045985 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207500455 0 0) 0x7f16640d0c50 con 0x7f1664160e60 2023-11-28T11:11:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16693f9700 1 -- 172.21.15.83:0/1425045985 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640cda40 con 0x7f16640b5170 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16693f9700 1 --2- 172.21.15.83:0/1425045985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b5170 0x7f16640d1410 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f166408bdc0 tx=0x7f165c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6064f6ea078ac8c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16637fe700 1 -- 172.21.15.83:0/1425045985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00e070 con 0x7f16640b5170 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16637fe700 1 -- 172.21.15.83:0/1425045985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c00bbd0 con 0x7f16640b5170 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f16637fe700 1 -- 172.21.15.83:0/1425045985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c012430 con 0x7f16640b5170 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 -- 172.21.15.83:0/1425045985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b5170 msgr2=0x7f16640d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 --2- 172.21.15.83:0/1425045985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b5170 0x7f16640d1410 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f166408bdc0 tx=0x7f165c00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 -- 172.21.15.83:0/1425045985 shutdown_connections 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 --2- 172.21.15.83:0/1425045985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b5170 0x7f16640d1410 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 --2- 172.21.15.83:0/1425045985 >> v2:172.21.15.90:3300/0 conn(0x7f16640b88d0 0x7f16640cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 -- 172.21.15.83:0/1425045985 >> 172.21.15.83:0/1425045985 conn(0x7f166406d030 msgr2=0x7f16640d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 -- 172.21.15.83:0/1425045985 shutdown_connections 2023-11-28T11:11:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.354+0000 7f166ae5c700 1 -- 172.21.15.83:0/1425045985 wait complete. 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 Processor -- start 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 -- start start 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16640b5170 0x7f1664151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 0x7f166415f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1664160e60 0x7f16640c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641f1de0 con 0x7f16640b88d0 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166415d2d0 con 0x7f16640b5170 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.356+0000 7f166ae5c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c7690 con 0x7f1664160e60 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 0x7f166415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 0x7f166415f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52770/0 (socket says 172.21.15.83:52770) 2023-11-28T11:11:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 -- 172.21.15.83:0/654088428 learned_addr learned my addr 172.21.15.83:0/654088428 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1668bf8700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16640b5170 0x7f1664151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f16693f9700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1664160e60 0x7f16640c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 -- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1664160e60 msgr2=0x7f16640c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1664160e60 0x7f16640c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 -- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16640b5170 msgr2=0x7f1664151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16640b5170 0x7f1664151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 -- 172.21.15.83:0/654088428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c009a70 con 0x7f16640b88d0 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1668bf8700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16640b5170 0x7f1664151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f16693f9700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1664160e60 0x7f16640c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f1663fff700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 0x7f166415f290 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f165000c750 tx=0x7f165000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650004c80 con 0x7f16640b88d0 2023-11-28T11:11:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1650004e00 con 0x7f16640b88d0 2023-11-28T11:11:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16641612d0 con 0x7f16640b88d0 2023-11-28T11:11:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650007310 con 0x7f16640b88d0 2023-11-28T11:11:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.357+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16640d0ef0 con 0x7f16640b88d0 2023-11-28T11:11:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.358+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16640cdd20 con 0x7f16640b88d0 2023-11-28T11:11:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.359+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f165001a020 con 0x7f16640b88d0 2023-11-28T11:11:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.359+0000 7f16617fa700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f16480682f0 0x7f164806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.359+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1650085a00 con 0x7f16640b88d0 2023-11-28T11:11:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.359+0000 7f1668bf8700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f16480682f0 0x7f164806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.359+0000 7f1668bf8700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f16480682f0 0x7f164806a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1658006fd0 tx=0x7f1658008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.363+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1650052120 con 0x7f16640b88d0 2023-11-28T11:11:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.620+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f16640c7b80 con 0x7f16640b88d0 2023-11-28T11:11:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.621+0000 7f16617fa700 1 -- 172.21.15.83:0/654088428 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1527 (secure 0 0 0) 0x7f1650055740 con 0x7f16640b88d0 2023-11-28T11:11:29.623 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:29.623 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":191,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1701169708,"num_in_osds":8,"osd_in_since":1701169700,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":79896576,"bytes_avail":773014216704,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":33,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6231}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-11-28T11:11:13.602547+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T11:11:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.622+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f16480682f0 msgr2=0x7f164806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.622+0000 7f166ae5c700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f16480682f0 0x7f164806a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1658006fd0 tx=0x7f1658008040 comp rx=0 tx=0).stop 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.622+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 msgr2=0x7f166415f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.622+0000 7f166ae5c700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 0x7f166415f290 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f165000c750 tx=0x7f165000cb10 comp rx=0 tx=0).stop 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 shutdown_connections 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1664160e60 0x7f16640c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16640b5170 0x7f1664151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f16480682f0 0x7f164806a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 --2- 172.21.15.83:0/654088428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640b88d0 0x7f166415f290 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 >> 172.21.15.83:0/654088428 conn(0x7f166406d030 msgr2=0x7f16640b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:29.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 shutdown_connections 2023-11-28T11:11:29.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.623+0000 7f166ae5c700 1 -- 172.21.15.83:0/654088428 wait complete. 2023-11-28T11:11:29.636 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T11:11:29.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:29.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.754+0000 7fb5bd12a700 1 Processor -- start 2023-11-28T11:11:29.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.754+0000 7fb5bd12a700 1 -- start start 2023-11-28T11:11:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5bd12a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5b80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5bd12a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b806ff80 con 0x7fb5b806cf90 2023-11-28T11:11:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5bd12a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b8161450 con 0x7fb5b815cc20 2023-11-28T11:11:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5bd12a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80ce630 con 0x7fb5b815c850 2023-11-28T11:11:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b759e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb5b806cf90 0x7fb5b806f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50640/0 (socket says 172.21.15.83:50640) 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b759e700 1 -- 172.21.15.83:0/2458215569 learned_addr learned my addr 172.21.15.83:0/2458215569 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838992772 0 0) 0x7fb5b806ff80 con 0x7fb5b806cf90 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5b80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a00035b0 con 0x7fb5b806cf90 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046882637 0 0) 0x7fb5a00035b0 con 0x7fb5b806cf90 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 msgr2=0x7fb5b80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5b80ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 >> v1:172.21.15.90:6789/0 conn(0x7fb5b815cc20 legacy=0x7fb5b806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b81608d0 con 0x7fb5b806cf90 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.755+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fb5b00019f0 con 0x7fb5b806cf90 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 >> v1:172.21.15.83:6789/0 conn(0x7fb5b806cf90 legacy=0x7fb5b806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5a0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a0004d90 0x7fb5a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80ce630 con 0x7fb5a0003a30 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a000bb20 con 0x7fb5a0004d90 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a000bce0 con 0x7fb5b815c850 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5a0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b659c700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b759e700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a0004d90 0x7fb5a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 msgr2=0x7fb5a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a0004d90 msgr2=0x7fb5a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5a0004d90 0x7fb5a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/2458215569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a00035b0 con 0x7fb5a0003a30 2023-11-28T11:11:29.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b659c700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b815c850 0x7fb5a0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5a0003e40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009b60 tx=0x7fb5a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af1834ae000b890 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00e6c0 con 0x7fb5a0003a30 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac00e820 con 0x7fb5a0003a30 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.756+0000 7fb5b5d9b700 1 -- 172.21.15.83:0/2458215569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00eb90 con 0x7fb5a0003a30 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5bd12a700 1 -- 172.21.15.83:0/2458215569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 msgr2=0x7fb5a0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/2458215569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5a0003e40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009b60 tx=0x7fb5a0005880 comp rx=0 tx=0).stop 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5b759e700 1 -- 172.21.15.83:0/2458215569 reap_dead start 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5bd12a700 1 -- 172.21.15.83:0/2458215569 shutdown_connections 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5bd12a700 1 -- 172.21.15.83:0/2458215569 >> 172.21.15.83:0/2458215569 conn(0x7fb5b80cd960 msgr2=0x7fb5b80cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5bd12a700 1 -- 172.21.15.83:0/2458215569 shutdown_connections 2023-11-28T11:11:29.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.757+0000 7fb5bd12a700 1 -- 172.21.15.83:0/2458215569 wait complete. 2023-11-28T11:11:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.759+0000 7fb5bd12a700 1 Processor -- start 2023-11-28T11:11:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.759+0000 7fb5bd12a700 1 -- start start 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5b815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b814d1a0 0x7fb5b8158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5b80b5c00 0x7fb5b80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80b6590 con 0x7fb5a0003a30 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80ad8e0 con 0x7fb5b80b5c00 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b815c060 con 0x7fb5b814d1a0 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5b815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5b815b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52788/0 (socket says 172.21.15.83:52788) 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/3757784318 learned_addr learned my addr 172.21.15.83:0/3757784318 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/3757784318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b814d1a0 msgr2=0x7fb5b8158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b814d1a0 0x7fb5b8158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/3757784318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5b80b5c00 msgr2=0x7fb5b80b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5b80b5c00 0x7fb5b80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 -- 172.21.15.83:0/3757784318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8160e90 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5b6d9d700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5b815b1e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000baf0 tx=0x7fb5ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac021070 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac004550 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b8160bb0 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00e6c0 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.760+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b8161170 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.761+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b8074320 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.762+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5ac02e030 con 0x7fb5a0003a30 2023-11-28T11:11:29.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.762+0000 7fb59f7fe700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5a40682f0 0x7fb5a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:29.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.762+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb5ac087090 con 0x7fb5a0003a30 2023-11-28T11:11:29.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.762+0000 7fb5b659c700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5a40682f0 0x7fb5a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:29.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.763+0000 7fb5b659c700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5a40682f0 0x7fb5a406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8157b40 tx=0x7fb5a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:29.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.766+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5ac0b85e0 con 0x7fb5a0003a30 2023-11-28T11:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.943+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5b80d1b30 con 0x7fb5a40682f0 2023-11-28T11:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.951+0000 7fb59f7fe700 1 -- 172.21.15.83:0/3757784318 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311604 (secure 0 0 0) 0x7fb5b80d1b30 con 0x7fb5a40682f0 2023-11-28T11:11:29.956 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5a40682f0 msgr2=0x7fb5a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5a40682f0 0x7fb5a406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8157b40 tx=0x7fb5a8009040 comp rx=0 tx=0).stop 2023-11-28T11:11:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 msgr2=0x7fb5b815b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5b815b1e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000baf0 tx=0x7fb5ac0047c0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 shutdown_connections 2023-11-28T11:11:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5b80b5c00 0x7fb5b80b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb5a40682f0 0x7fb5a406a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0003a30 0x7fb5b815b1e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 --2- 172.21.15.83:0/3757784318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5b814d1a0 0x7fb5b8158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 >> 172.21.15.83:0/3757784318 conn(0x7fb5b80cd960 msgr2=0x7fb5b80d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 shutdown_connections 2023-11-28T11:11:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:29.955+0000 7fb5bd12a700 1 -- 172.21.15.83:0/3757784318 wait complete. 2023-11-28T11:11:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:11:29.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2023-11-28T11:11:29.605474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":79616,"kb_used_data":7168,"kb_used_omap":0,"kb_used_meta":72384,"kb_avail":754895104,"statfs":{"total":773094113280,"available":773012586496,"internally_reserved":0,"allocated":7340032,"data_stored":4036906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74121216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001074"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201525+0000","last_change":"2023-11-28T11:11:24.411949+0000","last_active":"2023-11-28T11:11:27.201525+0000","last_peered":"2023-11-28T11:11:27.201525+0000","last_clean":"2023-11-28T11:11:27.201525+0000","last_became_active":"2023-11-28T11:11:24.409554+0000","last_became_peered":"2023-11-28T11:11:24.409554+0000","last_unstale":"2023-11-28T11:11:27.201525+0000","last_undegraded":"2023-11-28T11:11:27.201525+0000","last_fullsized":"2023-11-28T11:11:27.201525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:40.209892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388821+0000","last_change":"2023-11-28T11:11:24.409999+0000","last_active":"2023-11-28T11:11:25.388821+0000","last_peered":"2023-11-28T11:11:25.388821+0000","last_clean":"2023-11-28T11:11:25.388821+0000","last_became_active":"2023-11-28T11:11:24.409139+0000","last_became_peered":"2023-11-28T11:11:24.409139+0000","last_unstale":"2023-11-28T11:11:25.388821+0000","last_undegraded":"2023-11-28T11:11:25.388821+0000","last_fullsized":"2023-11-28T11:11:25.388821+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:01.031365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200218+0000","last_change":"2023-11-28T11:11:24.395340+0000","last_active":"2023-11-28T11:11:27.200218+0000","last_peered":"2023-11-28T11:11:27.200218+0000","last_clean":"2023-11-28T11:11:27.200218+0000","last_became_active":"2023-11-28T11:11:24.395094+0000","last_became_peered":"2023-11-28T11:11:24.395094+0000","last_unstale":"2023-11-28T11:11:27.200218+0000","last_undegraded":"2023-11-28T11:11:27.200218+0000","last_fullsized":"2023-11-28T11:11:27.200218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:24.943756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388485+0000","last_change":"2023-11-28T11:11:24.405522+0000","last_active":"2023-11-28T11:11:25.388485+0000","last_peered":"2023-11-28T11:11:25.388485+0000","last_clean":"2023-11-28T11:11:25.388485+0000","last_became_active":"2023-11-28T11:11:24.405145+0000","last_became_peered":"2023-11-28T11:11:24.405145+0000","last_unstale":"2023-11-28T11:11:25.388485+0000","last_undegraded":"2023-11-28T11:11:25.388485+0000","last_fullsized":"2023-11-28T11:11:25.388485+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:25.592467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387770+0000","last_change":"2023-11-28T11:11:24.403856+0000","last_active":"2023-11-28T11:11:25.387770+0000","last_peered":"2023-11-28T11:11:25.387770+0000","last_clean":"2023-11-28T11:11:25.387770+0000","last_became_active":"2023-11-28T11:11:24.403625+0000","last_became_peered":"2023-11-28T11:11:24.403625+0000","last_unstale":"2023-11-28T11:11:25.387770+0000","last_undegraded":"2023-11-28T11:11:25.387770+0000","last_fullsized":"2023-11-28T11:11:25.387770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:50.823320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388381+0000","last_change":"2023-11-28T11:11:24.407857+0000","last_active":"2023-11-28T11:11:25.388381+0000","last_peered":"2023-11-28T11:11:25.388381+0000","last_clean":"2023-11-28T11:11:25.388381+0000","last_became_active":"2023-11-28T11:11:24.405812+0000","last_became_peered":"2023-11-28T11:11:24.405812+0000","last_unstale":"2023-11-28T11:11:25.388381+0000","last_undegraded":"2023-11-28T11:11:25.388381+0000","last_fullsized":"2023-11-28T11:11:25.388381+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:18.535307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806226+0000","last_change":"2023-11-28T11:11:24.404376+0000","last_active":"2023-11-28T11:11:25.806226+0000","last_peered":"2023-11-28T11:11:25.806226+0000","last_clean":"2023-11-28T11:11:25.806226+0000","last_became_active":"2023-11-28T11:11:24.404099+0000","last_became_peered":"2023-11-28T11:11:24.404099+0000","last_unstale":"2023-11-28T11:11:25.806226+0000","last_undegraded":"2023-11-28T11:11:25.806226+0000","last_fullsized":"2023-11-28T11:11:25.806226+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:20.945993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388241+0000","last_change":"2023-11-28T11:11:24.409241+0000","last_active":"2023-11-28T11:11:25.388241+0000","last_peered":"2023-11-28T11:11:25.388241+0000","last_clean":"2023-11-28T11:11:25.388241+0000","last_became_active":"2023-11-28T11:11:24.408977+0000","last_became_peered":"2023-11-28T11:11:24.408977+0000","last_unstale":"2023-11-28T11:11:25.388241+0000","last_undegraded":"2023-11-28T11:11:25.388241+0000","last_fullsized":"2023-11-28T11:11:25.388241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:42.596380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388108+0000","last_change":"2023-11-28T11:11:24.396619+0000","last_active":"2023-11-28T11:11:25.388108+0000","last_peered":"2023-11-28T11:11:25.388108+0000","last_clean":"2023-11-28T11:11:25.388108+0000","last_became_active":"2023-11-28T11:11:24.396319+0000","last_became_peered":"2023-11-28T11:11:24.396319+0000","last_unstale":"2023-11-28T11:11:25.388108+0000","last_undegraded":"2023-11-28T11:11:25.388108+0000","last_fullsized":"2023-11-28T11:11:25.388108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:11.227361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201780+0000","last_change":"2023-11-28T11:11:24.396844+0000","last_active":"2023-11-28T11:11:27.201780+0000","last_peered":"2023-11-28T11:11:27.201780+0000","last_clean":"2023-11-28T11:11:27.201780+0000","last_became_active":"2023-11-28T11:11:24.396569+0000","last_became_peered":"2023-11-28T11:11:24.396569+0000","last_unstale":"2023-11-28T11:11:27.201780+0000","last_undegraded":"2023-11-28T11:11:27.201780+0000","last_fullsized":"2023-11-28T11:11:27.201780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:04.352002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201311+0000","last_change":"2023-11-28T11:11:24.402673+0000","last_active":"2023-11-28T11:11:27.201311+0000","last_peered":"2023-11-28T11:11:27.201311+0000","last_clean":"2023-11-28T11:11:27.201311+0000","last_became_active":"2023-11-28T11:11:24.402386+0000","last_became_peered":"2023-11-28T11:11:24.402386+0000","last_unstale":"2023-11-28T11:11:27.201311+0000","last_undegraded":"2023-11-28T11:11:27.201311+0000","last_fullsized":"2023-11-28T11:11:27.201311+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:06.857506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388552+0000","last_change":"2023-11-28T11:11:24.398769+0000","last_active":"2023-11-28T11:11:25.388552+0000","last_peered":"2023-11-28T11:11:25.388552+0000","last_clean":"2023-11-28T11:11:25.388552+0000","last_became_active":"2023-11-28T11:11:24.397605+0000","last_became_peered":"2023-11-28T11:11:24.397605+0000","last_unstale":"2023-11-28T11:11:25.388552+0000","last_undegraded":"2023-11-28T11:11:25.388552+0000","last_fullsized":"2023-11-28T11:11:25.388552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:53.772762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804225+0000","last_change":"2023-11-28T11:11:24.405277+0000","last_active":"2023-11-28T11:11:25.804225+0000","last_peered":"2023-11-28T11:11:25.804225+0000","last_clean":"2023-11-28T11:11:25.804225+0000","last_became_active":"2023-11-28T11:11:24.404991+0000","last_became_peered":"2023-11-28T11:11:24.404991+0000","last_unstale":"2023-11-28T11:11:25.804225+0000","last_undegraded":"2023-11-28T11:11:25.804225+0000","last_fullsized":"2023-11-28T11:11:25.804225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:37.550783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387700+0000","last_change":"2023-11-28T11:11:24.404417+0000","last_active":"2023-11-28T11:11:25.387700+0000","last_peered":"2023-11-28T11:11:25.387700+0000","last_clean":"2023-11-28T11:11:25.387700+0000","last_became_active":"2023-11-28T11:11:24.404162+0000","last_became_peered":"2023-11-28T11:11:24.404162+0000","last_unstale":"2023-11-28T11:11:25.387700+0000","last_undegraded":"2023-11-28T11:11:25.387700+0000","last_fullsized":"2023-11-28T11:11:25.387700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:35.756145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804034+0000","last_change":"2023-11-28T11:11:24.397418+0000","last_active":"2023-11-28T11:11:25.804034+0000","last_peered":"2023-11-28T11:11:25.804034+0000","last_clean":"2023-11-28T11:11:25.804034+0000","last_became_active":"2023-11-28T11:11:24.397166+0000","last_became_peered":"2023-11-28T11:11:24.397166+0000","last_unstale":"2023-11-28T11:11:25.804034+0000","last_undegraded":"2023-11-28T11:11:25.804034+0000","last_fullsized":"2023-11-28T11:11:25.804034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:26.608370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201161+0000","last_change":"2023-11-28T11:11:24.412720+0000","last_active":"2023-11-28T11:11:27.201161+0000","last_peered":"2023-11-28T11:11:27.201161+0000","last_clean":"2023-11-28T11:11:27.201161+0000","last_became_active":"2023-11-28T11:11:24.412463+0000","last_became_peered":"2023-11-28T11:11:24.412463+0000","last_unstale":"2023-11-28T11:11:27.201161+0000","last_undegraded":"2023-11-28T11:11:27.201161+0000","last_fullsized":"2023-11-28T11:11:27.201161+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:25.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388155+0000","last_change":"2023-11-28T11:11:24.394251+0000","last_active":"2023-11-28T11:11:25.388155+0000","last_peered":"2023-11-28T11:11:25.388155+0000","last_clean":"2023-11-28T11:11:25.388155+0000","last_became_active":"2023-11-28T11:11:24.394006+0000","last_became_peered":"2023-11-28T11:11:24.394006+0000","last_unstale":"2023-11-28T11:11:25.388155+0000","last_undegraded":"2023-11-28T11:11:25.388155+0000","last_fullsized":"2023-11-28T11:11:25.388155+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:55:57.581783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388499+0000","last_change":"2023-11-28T11:11:24.412428+0000","last_active":"2023-11-28T11:11:25.388499+0000","last_peered":"2023-11-28T11:11:25.388499+0000","last_clean":"2023-11-28T11:11:25.388499+0000","last_became_active":"2023-11-28T11:11:24.412152+0000","last_became_peered":"2023-11-28T11:11:24.412152+0000","last_unstale":"2023-11-28T11:11:25.388499+0000","last_undegraded":"2023-11-28T11:11:25.388499+0000","last_fullsized":"2023-11-28T11:11:25.388499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:45.975342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388703+0000","last_change":"2023-11-28T11:11:24.409100+0000","last_active":"2023-11-28T11:11:25.388703+0000","last_peered":"2023-11-28T11:11:25.388703+0000","last_clean":"2023-11-28T11:11:25.388703+0000","last_became_active":"2023-11-28T11:11:24.408794+0000","last_became_peered":"2023-11-28T11:11:24.408794+0000","last_unstale":"2023-11-28T11:11:25.388703+0000","last_undegraded":"2023-11-28T11:11:25.388703+0000","last_fullsized":"2023-11-28T11:11:25.388703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:52.656366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804213+0000","last_change":"2023-11-28T11:11:24.400243+0000","last_active":"2023-11-28T11:11:25.804213+0000","last_peered":"2023-11-28T11:11:25.804213+0000","last_clean":"2023-11-28T11:11:25.804213+0000","last_became_active":"2023-11-28T11:11:24.399969+0000","last_became_peered":"2023-11-28T11:11:24.399969+0000","last_unstale":"2023-11-28T11:11:25.804213+0000","last_undegraded":"2023-11-28T11:11:25.804213+0000","last_fullsized":"2023-11-28T11:11:25.804213+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:28.573445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201299+0000","last_change":"2023-11-28T11:11:24.397089+0000","last_active":"2023-11-28T11:11:27.201299+0000","last_peered":"2023-11-28T11:11:27.201299+0000","last_clean":"2023-11-28T11:11:27.201299+0000","last_became_active":"2023-11-28T11:11:24.396624+0000","last_became_peered":"2023-11-28T11:11:24.396624+0000","last_unstale":"2023-11-28T11:11:27.201299+0000","last_undegraded":"2023-11-28T11:11:27.201299+0000","last_fullsized":"2023-11-28T11:11:27.201299+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:13.808465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201482+0000","last_change":"2023-11-28T11:11:24.396407+0000","last_active":"2023-11-28T11:11:27.201482+0000","last_peered":"2023-11-28T11:11:27.201482+0000","last_clean":"2023-11-28T11:11:27.201482+0000","last_became_active":"2023-11-28T11:11:24.396140+0000","last_became_peered":"2023-11-28T11:11:24.396140+0000","last_unstale":"2023-11-28T11:11:27.201482+0000","last_undegraded":"2023-11-28T11:11:27.201482+0000","last_fullsized":"2023-11-28T11:11:27.201482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:20:22.499380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201131+0000","last_change":"2023-11-28T11:11:24.405692+0000","last_active":"2023-11-28T11:11:27.201131+0000","last_peered":"2023-11-28T11:11:27.201131+0000","last_clean":"2023-11-28T11:11:27.201131+0000","last_became_active":"2023-11-28T11:11:24.405422+0000","last_became_peered":"2023-11-28T11:11:24.405422+0000","last_unstale":"2023-11-28T11:11:27.201131+0000","last_undegraded":"2023-11-28T11:11:27.201131+0000","last_fullsized":"2023-11-28T11:11:27.201131+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:10.665053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201073+0000","last_change":"2023-11-28T11:11:24.413275+0000","last_active":"2023-11-28T11:11:27.201073+0000","last_peered":"2023-11-28T11:11:27.201073+0000","last_clean":"2023-11-28T11:11:27.201073+0000","last_became_active":"2023-11-28T11:11:24.413006+0000","last_became_peered":"2023-11-28T11:11:24.413006+0000","last_unstale":"2023-11-28T11:11:27.201073+0000","last_undegraded":"2023-11-28T11:11:27.201073+0000","last_fullsized":"2023-11-28T11:11:27.201073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:13.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201795+0000","last_change":"2023-11-28T11:11:24.410281+0000","last_active":"2023-11-28T11:11:27.201795+0000","last_peered":"2023-11-28T11:11:27.201795+0000","last_clean":"2023-11-28T11:11:27.201795+0000","last_became_active":"2023-11-28T11:11:24.410009+0000","last_became_peered":"2023-11-28T11:11:24.410009+0000","last_unstale":"2023-11-28T11:11:27.201795+0000","last_undegraded":"2023-11-28T11:11:27.201795+0000","last_fullsized":"2023-11-28T11:11:27.201795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:43.598038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806633+0000","last_change":"2023-11-28T11:11:24.412179+0000","last_active":"2023-11-28T11:11:25.806633+0000","last_peered":"2023-11-28T11:11:25.806633+0000","last_clean":"2023-11-28T11:11:25.806633+0000","last_became_active":"2023-11-28T11:11:24.411887+0000","last_became_peered":"2023-11-28T11:11:24.411887+0000","last_unstale":"2023-11-28T11:11:25.806633+0000","last_undegraded":"2023-11-28T11:11:25.806633+0000","last_fullsized":"2023-11-28T11:11:25.806633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:56.687482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200275+0000","last_change":"2023-11-28T11:11:24.403340+0000","last_active":"2023-11-28T11:11:27.200275+0000","last_peered":"2023-11-28T11:11:27.200275+0000","last_clean":"2023-11-28T11:11:27.200275+0000","last_became_active":"2023-11-28T11:11:24.402878+0000","last_became_peered":"2023-11-28T11:11:24.402878+0000","last_unstale":"2023-11-28T11:11:27.200275+0000","last_undegraded":"2023-11-28T11:11:27.200275+0000","last_fullsized":"2023-11-28T11:11:27.200275+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:52.798672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201535+0000","last_change":"2023-11-28T11:11:24.397777+0000","last_active":"2023-11-28T11:11:27.201535+0000","last_peered":"2023-11-28T11:11:27.201535+0000","last_clean":"2023-11-28T11:11:27.201535+0000","last_became_active":"2023-11-28T11:11:24.397497+0000","last_became_peered":"2023-11-28T11:11:24.397497+0000","last_unstale":"2023-11-28T11:11:27.201535+0000","last_undegraded":"2023-11-28T11:11:27.201535+0000","last_fullsized":"2023-11-28T11:11:27.201535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:40.209892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200078+0000","last_change":"2023-11-28T11:11:24.404569+0000","last_active":"2023-11-28T11:11:27.200078+0000","last_peered":"2023-11-28T11:11:27.200078+0000","last_clean":"2023-11-28T11:11:27.200078+0000","last_became_active":"2023-11-28T11:11:24.404324+0000","last_became_peered":"2023-11-28T11:11:24.404324+0000","last_unstale":"2023-11-28T11:11:27.200078+0000","last_undegraded":"2023-11-28T11:11:27.200078+0000","last_fullsized":"2023-11-28T11:11:27.200078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:00.415431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806369+0000","last_change":"2023-11-28T11:11:24.406440+0000","last_active":"2023-11-28T11:11:25.806369+0000","last_peered":"2023-11-28T11:11:25.806369+0000","last_clean":"2023-11-28T11:11:25.806369+0000","last_became_active":"2023-11-28T11:11:24.406153+0000","last_became_peered":"2023-11-28T11:11:24.406153+0000","last_unstale":"2023-11-28T11:11:25.806369+0000","last_undegraded":"2023-11-28T11:11:25.806369+0000","last_fullsized":"2023-11-28T11:11:25.806369+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:25.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806497+0000","last_change":"2023-11-28T11:11:24.389706+0000","last_active":"2023-11-28T11:11:25.806497+0000","last_peered":"2023-11-28T11:11:25.806497+0000","last_clean":"2023-11-28T11:11:25.806497+0000","last_became_active":"2023-11-28T11:11:24.389407+0000","last_became_peered":"2023-11-28T11:11:24.389407+0000","last_unstale":"2023-11-28T11:11:25.806497+0000","last_undegraded":"2023-11-28T11:11:25.806497+0000","last_fullsized":"2023-11-28T11:11:25.806497+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:27.262120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388214+0000","last_change":"2023-11-28T11:11:24.411153+0000","last_active":"2023-11-28T11:11:25.388214+0000","last_peered":"2023-11-28T11:11:25.388214+0000","last_clean":"2023-11-28T11:11:25.388214+0000","last_became_active":"2023-11-28T11:11:24.410909+0000","last_became_peered":"2023-11-28T11:11:24.410909+0000","last_unstale":"2023-11-28T11:11:25.388214+0000","last_undegraded":"2023-11-28T11:11:25.388214+0000","last_fullsized":"2023-11-28T11:11:25.388214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:04.527247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201543+0000","last_change":"2023-11-28T11:11:24.409531+0000","last_active":"2023-11-28T11:11:27.201543+0000","last_peered":"2023-11-28T11:11:27.201543+0000","last_clean":"2023-11-28T11:11:27.201543+0000","last_became_active":"2023-11-28T11:11:24.409240+0000","last_became_peered":"2023-11-28T11:11:24.409240+0000","last_unstale":"2023-11-28T11:11:27.201543+0000","last_undegraded":"2023-11-28T11:11:27.201543+0000","last_fullsized":"2023-11-28T11:11:27.201543+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:25.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.416496+0000","last_change":"2023-11-28T11:11:24.008585+0000","last_active":"2023-11-28T11:11:24.416496+0000","last_peered":"2023-11-28T11:11:24.416496+0000","last_clean":"2023-11-28T11:11:24.416496+0000","last_became_active":"2023-11-28T11:11:24.008309+0000","last_became_peered":"2023-11-28T11:11:24.008309+0000","last_unstale":"2023-11-28T11:11:24.416496+0000","last_undegraded":"2023-11-28T11:11:24.416496+0000","last_fullsized":"2023-11-28T11:11:24.416496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:51.009482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388212+0000","last_change":"2023-11-28T11:11:24.389701+0000","last_active":"2023-11-28T11:11:25.388212+0000","last_peered":"2023-11-28T11:11:25.388212+0000","last_clean":"2023-11-28T11:11:25.388212+0000","last_became_active":"2023-11-28T11:11:24.389461+0000","last_became_peered":"2023-11-28T11:11:24.389461+0000","last_unstale":"2023-11-28T11:11:25.388212+0000","last_undegraded":"2023-11-28T11:11:25.388212+0000","last_fullsized":"2023-11-28T11:11:25.388212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:58.121727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401532+0000","last_change":"2023-11-28T11:11:24.001889+0000","last_active":"2023-11-28T11:11:24.401532+0000","last_peered":"2023-11-28T11:11:24.401532+0000","last_clean":"2023-11-28T11:11:24.401532+0000","last_became_active":"2023-11-28T11:11:24.001614+0000","last_became_peered":"2023-11-28T11:11:24.001614+0000","last_unstale":"2023-11-28T11:11:24.401532+0000","last_undegraded":"2023-11-28T11:11:24.401532+0000","last_fullsized":"2023-11-28T11:11:24.401532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:03.348424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388601+0000","last_change":"2023-11-28T11:11:24.408287+0000","last_active":"2023-11-28T11:11:25.388601+0000","last_peered":"2023-11-28T11:11:25.388601+0000","last_clean":"2023-11-28T11:11:25.388601+0000","last_became_active":"2023-11-28T11:11:24.406043+0000","last_became_peered":"2023-11-28T11:11:24.406043+0000","last_unstale":"2023-11-28T11:11:25.388601+0000","last_undegraded":"2023-11-28T11:11:25.388601+0000","last_fullsized":"2023-11-28T11:11:25.388601+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:15.559457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.399451+0000","last_change":"2023-11-28T11:11:24.006533+0000","last_active":"2023-11-28T11:11:24.399451+0000","last_peered":"2023-11-28T11:11:24.399451+0000","last_clean":"2023-11-28T11:11:24.399451+0000","last_became_active":"2023-11-28T11:11:24.005330+0000","last_became_peered":"2023-11-28T11:11:24.005330+0000","last_unstale":"2023-11-28T11:11:24.399451+0000","last_undegraded":"2023-11-28T11:11:24.399451+0000","last_fullsized":"2023-11-28T11:11:24.399451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:12.037178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201398+0000","last_change":"2023-11-28T11:11:24.412461+0000","last_active":"2023-11-28T11:11:27.201398+0000","last_peered":"2023-11-28T11:11:27.201398+0000","last_clean":"2023-11-28T11:11:27.201398+0000","last_became_active":"2023-11-28T11:11:24.409781+0000","last_became_peered":"2023-11-28T11:11:24.409781+0000","last_unstale":"2023-11-28T11:11:27.201398+0000","last_undegraded":"2023-11-28T11:11:27.201398+0000","last_fullsized":"2023-11-28T11:11:27.201398+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:20.945993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.414283+0000","last_change":"2023-11-28T11:11:24.003944+0000","last_active":"2023-11-28T11:11:24.414283+0000","last_peered":"2023-11-28T11:11:24.414283+0000","last_clean":"2023-11-28T11:11:24.414283+0000","last_became_active":"2023-11-28T11:11:24.003537+0000","last_became_peered":"2023-11-28T11:11:24.003537+0000","last_unstale":"2023-11-28T11:11:24.414283+0000","last_undegraded":"2023-11-28T11:11:24.414283+0000","last_fullsized":"2023-11-28T11:11:24.414283+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:40.682923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806435+0000","last_change":"2023-11-28T11:11:24.400254+0000","last_active":"2023-11-28T11:11:25.806435+0000","last_peered":"2023-11-28T11:11:25.806435+0000","last_clean":"2023-11-28T11:11:25.806435+0000","last_became_active":"2023-11-28T11:11:24.395751+0000","last_became_peered":"2023-11-28T11:11:24.395751+0000","last_unstale":"2023-11-28T11:11:25.806435+0000","last_undegraded":"2023-11-28T11:11:25.806435+0000","last_fullsized":"2023-11-28T11:11:25.806435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:13.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.396818+0000","last_change":"2023-11-28T11:11:24.005782+0000","last_active":"2023-11-28T11:11:24.396818+0000","last_peered":"2023-11-28T11:11:24.396818+0000","last_clean":"2023-11-28T11:11:24.396818+0000","last_became_active":"2023-11-28T11:11:24.005497+0000","last_became_peered":"2023-11-28T11:11:24.005497+0000","last_unstale":"2023-11-28T11:11:24.396818+0000","last_undegraded":"2023-11-28T11:11:24.396818+0000","last_fullsized":"2023-11-28T11:11:24.396818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:08.956676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201405+0000","last_change":"2023-11-28T11:11:24.407156+0000","last_active":"2023-11-28T11:11:27.201405+0000","last_peered":"2023-11-28T11:11:27.201405+0000","last_clean":"2023-11-28T11:11:27.201405+0000","last_became_active":"2023-11-28T11:11:24.406819+0000","last_became_peered":"2023-11-28T11:11:24.406819+0000","last_unstale":"2023-11-28T11:11:27.201405+0000","last_undegraded":"2023-11-28T11:11:27.201405+0000","last_fullsized":"2023-11-28T11:11:27.201405+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:09.743552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.413902+0000","last_change":"2023-11-28T11:11:24.002284+0000","last_active":"2023-11-28T11:11:24.413902+0000","last_peered":"2023-11-28T11:11:24.413902+0000","last_clean":"2023-11-28T11:11:24.413902+0000","last_became_active":"2023-11-28T11:11:24.001981+0000","last_became_peered":"2023-11-28T11:11:24.001981+0000","last_unstale":"2023-11-28T11:11:24.413902+0000","last_undegraded":"2023-11-28T11:11:24.413902+0000","last_fullsized":"2023-11-28T11:11:24.413902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:49.819742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387264+0000","last_change":"2023-11-28T11:11:24.403553+0000","last_active":"2023-11-28T11:11:25.387264+0000","last_peered":"2023-11-28T11:11:25.387264+0000","last_clean":"2023-11-28T11:11:25.387264+0000","last_became_active":"2023-11-28T11:11:24.403273+0000","last_became_peered":"2023-11-28T11:11:24.403273+0000","last_unstale":"2023-11-28T11:11:25.387264+0000","last_undegraded":"2023-11-28T11:11:25.387264+0000","last_fullsized":"2023-11-28T11:11:25.387264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:04.090203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390263+0000","last_change":"2023-11-28T11:11:24.005594+0000","last_active":"2023-11-28T11:11:24.390263+0000","last_peered":"2023-11-28T11:11:24.390263+0000","last_clean":"2023-11-28T11:11:24.390263+0000","last_became_active":"2023-11-28T11:11:24.005372+0000","last_became_peered":"2023-11-28T11:11:24.005372+0000","last_unstale":"2023-11-28T11:11:24.390263+0000","last_undegraded":"2023-11-28T11:11:24.390263+0000","last_fullsized":"2023-11-28T11:11:24.390263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804078+0000","last_change":"2023-11-28T11:11:24.403292+0000","last_active":"2023-11-28T11:11:25.804078+0000","last_peered":"2023-11-28T11:11:25.804078+0000","last_clean":"2023-11-28T11:11:25.804078+0000","last_became_active":"2023-11-28T11:11:24.403029+0000","last_became_peered":"2023-11-28T11:11:24.403029+0000","last_unstale":"2023-11-28T11:11:25.804078+0000","last_undegraded":"2023-11-28T11:11:25.804078+0000","last_fullsized":"2023-11-28T11:11:25.804078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:34.701359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.383024+0000","last_change":"2023-11-28T11:11:23.998331+0000","last_active":"2023-11-28T11:11:24.383024+0000","last_peered":"2023-11-28T11:11:24.383024+0000","last_clean":"2023-11-28T11:11:24.383024+0000","last_became_active":"2023-11-28T11:11:23.997618+0000","last_became_peered":"2023-11-28T11:11:23.997618+0000","last_unstale":"2023-11-28T11:11:24.383024+0000","last_undegraded":"2023-11-28T11:11:24.383024+0000","last_fullsized":"2023-11-28T11:11:24.383024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:53.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806636+0000","last_change":"2023-11-28T11:11:24.401893+0000","last_active":"2023-11-28T11:11:25.806636+0000","last_peered":"2023-11-28T11:11:25.806636+0000","last_clean":"2023-11-28T11:11:25.806636+0000","last_became_active":"2023-11-28T11:11:24.401629+0000","last_became_peered":"2023-11-28T11:11:24.401629+0000","last_unstale":"2023-11-28T11:11:25.806636+0000","last_undegraded":"2023-11-28T11:11:25.806636+0000","last_fullsized":"2023-11-28T11:11:25.806636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:32.718969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390876+0000","last_change":"2023-11-28T11:11:24.006121+0000","last_active":"2023-11-28T11:11:24.390876+0000","last_peered":"2023-11-28T11:11:24.390876+0000","last_clean":"2023-11-28T11:11:24.390876+0000","last_became_active":"2023-11-28T11:11:24.005748+0000","last_became_peered":"2023-11-28T11:11:24.005748+0000","last_unstale":"2023-11-28T11:11:24.390876+0000","last_undegraded":"2023-11-28T11:11:24.390876+0000","last_fullsized":"2023-11-28T11:11:24.390876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:00.027787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201720+0000","last_change":"2023-11-28T11:11:24.413462+0000","last_active":"2023-11-28T11:11:27.201720+0000","last_peered":"2023-11-28T11:11:27.201720+0000","last_clean":"2023-11-28T11:11:27.201720+0000","last_became_active":"2023-11-28T11:11:24.411901+0000","last_became_peered":"2023-11-28T11:11:24.411901+0000","last_unstale":"2023-11-28T11:11:27.201720+0000","last_undegraded":"2023-11-28T11:11:27.201720+0000","last_fullsized":"2023-11-28T11:11:27.201720+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:15.559457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.402419+0000","last_change":"2023-11-28T11:11:24.002718+0000","last_active":"2023-11-28T11:11:24.402419+0000","last_peered":"2023-11-28T11:11:24.402419+0000","last_clean":"2023-11-28T11:11:24.402419+0000","last_became_active":"2023-11-28T11:11:24.002376+0000","last_became_peered":"2023-11-28T11:11:24.002376+0000","last_unstale":"2023-11-28T11:11:24.402419+0000","last_undegraded":"2023-11-28T11:11:24.402419+0000","last_fullsized":"2023-11-28T11:11:24.402419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388675+0000","last_change":"2023-11-28T11:11:24.402031+0000","last_active":"2023-11-28T11:11:25.388675+0000","last_peered":"2023-11-28T11:11:25.388675+0000","last_clean":"2023-11-28T11:11:25.388675+0000","last_became_active":"2023-11-28T11:11:24.401775+0000","last_became_peered":"2023-11-28T11:11:24.401775+0000","last_unstale":"2023-11-28T11:11:25.388675+0000","last_undegraded":"2023-11-28T11:11:25.388675+0000","last_fullsized":"2023-11-28T11:11:25.388675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:13.040756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.404264+0000","last_change":"2023-11-28T11:11:24.006140+0000","last_active":"2023-11-28T11:11:24.404264+0000","last_peered":"2023-11-28T11:11:24.404264+0000","last_clean":"2023-11-28T11:11:24.404264+0000","last_became_active":"2023-11-28T11:11:24.005908+0000","last_became_peered":"2023-11-28T11:11:24.005908+0000","last_unstale":"2023-11-28T11:11:24.404264+0000","last_undegraded":"2023-11-28T11:11:24.404264+0000","last_fullsized":"2023-11-28T11:11:24.404264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:06.471882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388114+0000","last_change":"2023-11-28T11:11:24.396415+0000","last_active":"2023-11-28T11:11:25.388114+0000","last_peered":"2023-11-28T11:11:25.388114+0000","last_clean":"2023-11-28T11:11:25.388114+0000","last_became_active":"2023-11-28T11:11:24.396147+0000","last_became_peered":"2023-11-28T11:11:24.396147+0000","last_unstale":"2023-11-28T11:11:25.388114+0000","last_undegraded":"2023-11-28T11:11:25.388114+0000","last_fullsized":"2023-11-28T11:11:25.388114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:04.662088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.409658+0000","last_change":"2023-11-28T11:11:23.997636+0000","last_active":"2023-11-28T11:11:24.409658+0000","last_peered":"2023-11-28T11:11:24.409658+0000","last_clean":"2023-11-28T11:11:24.409658+0000","last_became_active":"2023-11-28T11:11:23.997302+0000","last_became_peered":"2023-11-28T11:11:23.997302+0000","last_unstale":"2023-11-28T11:11:24.409658+0000","last_undegraded":"2023-11-28T11:11:24.409658+0000","last_fullsized":"2023-11-28T11:11:24.409658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:03.658510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201705+0000","last_change":"2023-11-28T11:11:24.404561+0000","last_active":"2023-11-28T11:11:27.201705+0000","last_peered":"2023-11-28T11:11:27.201705+0000","last_clean":"2023-11-28T11:11:27.201705+0000","last_became_active":"2023-11-28T11:11:24.404283+0000","last_became_peered":"2023-11-28T11:11:24.404283+0000","last_unstale":"2023-11-28T11:11:27.201705+0000","last_undegraded":"2023-11-28T11:11:27.201705+0000","last_fullsized":"2023-11-28T11:11:27.201705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:13.040756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.412464+0000","last_change":"2023-11-28T11:11:23.997769+0000","last_active":"2023-11-28T11:11:24.412464+0000","last_peered":"2023-11-28T11:11:24.412464+0000","last_clean":"2023-11-28T11:11:24.412464+0000","last_became_active":"2023-11-28T11:11:23.997513+0000","last_became_peered":"2023-11-28T11:11:23.997513+0000","last_unstale":"2023-11-28T11:11:24.412464+0000","last_undegraded":"2023-11-28T11:11:24.412464+0000","last_fullsized":"2023-11-28T11:11:24.412464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806350+0000","last_change":"2023-11-28T11:11:24.399918+0000","last_active":"2023-11-28T11:11:25.806350+0000","last_peered":"2023-11-28T11:11:25.806350+0000","last_clean":"2023-11-28T11:11:25.806350+0000","last_became_active":"2023-11-28T11:11:24.398006+0000","last_became_peered":"2023-11-28T11:11:24.398006+0000","last_unstale":"2023-11-28T11:11:25.806350+0000","last_undegraded":"2023-11-28T11:11:25.806350+0000","last_fullsized":"2023-11-28T11:11:25.806350+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:33.262768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401726+0000","last_change":"2023-11-28T11:11:24.002298+0000","last_active":"2023-11-28T11:11:24.401726+0000","last_peered":"2023-11-28T11:11:24.401726+0000","last_clean":"2023-11-28T11:11:24.401726+0000","last_became_active":"2023-11-28T11:11:24.002014+0000","last_became_peered":"2023-11-28T11:11:24.002014+0000","last_unstale":"2023-11-28T11:11:24.401726+0000","last_undegraded":"2023-11-28T11:11:24.401726+0000","last_fullsized":"2023-11-28T11:11:24.401726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:34.752567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.803999+0000","last_change":"2023-11-28T11:11:24.406640+0000","last_active":"2023-11-28T11:11:25.803999+0000","last_peered":"2023-11-28T11:11:25.803999+0000","last_clean":"2023-11-28T11:11:25.803999+0000","last_became_active":"2023-11-28T11:11:24.406387+0000","last_became_peered":"2023-11-28T11:11:24.406387+0000","last_unstale":"2023-11-28T11:11:25.803999+0000","last_undegraded":"2023-11-28T11:11:25.803999+0000","last_fullsized":"2023-11-28T11:11:25.803999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:48.019516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.385046+0000","last_change":"2023-11-28T11:11:24.001468+0000","last_active":"2023-11-28T11:11:24.385046+0000","last_peered":"2023-11-28T11:11:24.385046+0000","last_clean":"2023-11-28T11:11:24.385046+0000","last_became_active":"2023-11-28T11:11:24.001211+0000","last_became_peered":"2023-11-28T11:11:24.001211+0000","last_unstale":"2023-11-28T11:11:24.385046+0000","last_undegraded":"2023-11-28T11:11:24.385046+0000","last_fullsized":"2023-11-28T11:11:24.385046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:21.311957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387448+0000","last_change":"2023-11-28T11:11:24.395465+0000","last_active":"2023-11-28T11:11:25.387448+0000","last_peered":"2023-11-28T11:11:25.387448+0000","last_clean":"2023-11-28T11:11:25.387448+0000","last_became_active":"2023-11-28T11:11:24.395214+0000","last_became_peered":"2023-11-28T11:11:24.395214+0000","last_unstale":"2023-11-28T11:11:25.387448+0000","last_undegraded":"2023-11-28T11:11:25.387448+0000","last_fullsized":"2023-11-28T11:11:25.387448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:52.739555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415054+0000","last_change":"2023-11-28T11:11:24.008129+0000","last_active":"2023-11-28T11:11:24.415054+0000","last_peered":"2023-11-28T11:11:24.415054+0000","last_clean":"2023-11-28T11:11:24.415054+0000","last_became_active":"2023-11-28T11:11:24.007809+0000","last_became_peered":"2023-11-28T11:11:24.007809+0000","last_unstale":"2023-11-28T11:11:24.415054+0000","last_undegraded":"2023-11-28T11:11:24.415054+0000","last_fullsized":"2023-11-28T11:11:24.415054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:03.523669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804260+0000","last_change":"2023-11-28T11:11:24.401697+0000","last_active":"2023-11-28T11:11:25.804260+0000","last_peered":"2023-11-28T11:11:25.804260+0000","last_clean":"2023-11-28T11:11:25.804260+0000","last_became_active":"2023-11-28T11:11:24.401374+0000","last_became_peered":"2023-11-28T11:11:24.401374+0000","last_unstale":"2023-11-28T11:11:25.804260+0000","last_undegraded":"2023-11-28T11:11:25.804260+0000","last_fullsized":"2023-11-28T11:11:25.804260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.621600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415600+0000","last_change":"2023-11-28T11:11:24.003527+0000","last_active":"2023-11-28T11:11:24.415600+0000","last_peered":"2023-11-28T11:11:24.415600+0000","last_clean":"2023-11-28T11:11:24.415600+0000","last_became_active":"2023-11-28T11:11:24.003238+0000","last_became_peered":"2023-11-28T11:11:24.003238+0000","last_unstale":"2023-11-28T11:11:24.415600+0000","last_undegraded":"2023-11-28T11:11:24.415600+0000","last_fullsized":"2023-11-28T11:11:24.415600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:20.035165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806670+0000","last_change":"2023-11-28T11:11:24.400115+0000","last_active":"2023-11-28T11:11:25.806670+0000","last_peered":"2023-11-28T11:11:25.806670+0000","last_clean":"2023-11-28T11:11:25.806670+0000","last_became_active":"2023-11-28T11:11:24.395921+0000","last_became_peered":"2023-11-28T11:11:24.395921+0000","last_unstale":"2023-11-28T11:11:25.806670+0000","last_undegraded":"2023-11-28T11:11:25.806670+0000","last_fullsized":"2023-11-28T11:11:25.806670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:15.559457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.385231+0000","last_change":"2023-11-28T11:11:23.999779+0000","last_active":"2023-11-28T11:11:24.385231+0000","last_peered":"2023-11-28T11:11:24.385231+0000","last_clean":"2023-11-28T11:11:24.385231+0000","last_became_active":"2023-11-28T11:11:23.999526+0000","last_became_peered":"2023-11-28T11:11:23.999526+0000","last_unstale":"2023-11-28T11:11:24.385231+0000","last_undegraded":"2023-11-28T11:11:24.385231+0000","last_fullsized":"2023-11-28T11:11:24.385231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:28.037670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201286+0000","last_change":"2023-11-28T11:11:24.409840+0000","last_active":"2023-11-28T11:11:27.201286+0000","last_peered":"2023-11-28T11:11:27.201286+0000","last_clean":"2023-11-28T11:11:27.201286+0000","last_became_active":"2023-11-28T11:11:24.409407+0000","last_became_peered":"2023-11-28T11:11:24.409407+0000","last_unstale":"2023-11-28T11:11:27.201286+0000","last_undegraded":"2023-11-28T11:11:27.201286+0000","last_fullsized":"2023-11-28T11:11:27.201286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:20:22.499380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.384161+0000","last_change":"2023-11-28T11:11:24.001130+0000","last_active":"2023-11-28T11:11:24.384161+0000","last_peered":"2023-11-28T11:11:24.384161+0000","last_clean":"2023-11-28T11:11:24.384161+0000","last_became_active":"2023-11-28T11:11:24.000888+0000","last_became_peered":"2023-11-28T11:11:24.000888+0000","last_unstale":"2023-11-28T11:11:24.384161+0000","last_undegraded":"2023-11-28T11:11:24.384161+0000","last_fullsized":"2023-11-28T11:11:24.384161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:43:12.693701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201512+0000","last_change":"2023-11-28T11:11:24.413129+0000","last_active":"2023-11-28T11:11:27.201512+0000","last_peered":"2023-11-28T11:11:27.201512+0000","last_clean":"2023-11-28T11:11:27.201512+0000","last_became_active":"2023-11-28T11:11:24.412439+0000","last_became_peered":"2023-11-28T11:11:24.412439+0000","last_unstale":"2023-11-28T11:11:27.201512+0000","last_undegraded":"2023-11-28T11:11:27.201512+0000","last_fullsized":"2023-11-28T11:11:27.201512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:56.687482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.392498+0000","last_change":"2023-11-28T11:11:24.005822+0000","last_active":"2023-11-28T11:11:24.392498+0000","last_peered":"2023-11-28T11:11:24.392498+0000","last_clean":"2023-11-28T11:11:24.392498+0000","last_became_active":"2023-11-28T11:11:24.005558+0000","last_became_peered":"2023-11-28T11:11:24.005558+0000","last_unstale":"2023-11-28T11:11:24.392498+0000","last_undegraded":"2023-11-28T11:11:24.392498+0000","last_fullsized":"2023-11-28T11:11:24.392498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:10.349250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387781+0000","last_change":"2023-11-28T11:11:24.394292+0000","last_active":"2023-11-28T11:11:25.387781+0000","last_peered":"2023-11-28T11:11:25.387781+0000","last_clean":"2023-11-28T11:11:25.387781+0000","last_became_active":"2023-11-28T11:11:24.394024+0000","last_became_peered":"2023-11-28T11:11:24.394024+0000","last_unstale":"2023-11-28T11:11:25.387781+0000","last_undegraded":"2023-11-28T11:11:25.387781+0000","last_fullsized":"2023-11-28T11:11:25.387781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:21.038743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.383850+0000","last_change":"2023-11-28T11:11:23.996930+0000","last_active":"2023-11-28T11:11:24.383850+0000","last_peered":"2023-11-28T11:11:24.383850+0000","last_clean":"2023-11-28T11:11:24.383850+0000","last_became_active":"2023-11-28T11:11:23.996637+0000","last_became_peered":"2023-11-28T11:11:23.996637+0000","last_unstale":"2023-11-28T11:11:24.383850+0000","last_undegraded":"2023-11-28T11:11:24.383850+0000","last_fullsized":"2023-11-28T11:11:24.383850+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:36.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388500+0000","last_change":"2023-11-28T11:11:24.413366+0000","last_active":"2023-11-28T11:11:25.388500+0000","last_peered":"2023-11-28T11:11:25.388500+0000","last_clean":"2023-11-28T11:11:25.388500+0000","last_became_active":"2023-11-28T11:11:24.413074+0000","last_became_peered":"2023-11-28T11:11:24.413074+0000","last_unstale":"2023-11-28T11:11:25.388500+0000","last_undegraded":"2023-11-28T11:11:25.388500+0000","last_fullsized":"2023-11-28T11:11:25.388500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:10.665053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.391221+0000","last_change":"2023-11-28T11:11:24.005663+0000","last_active":"2023-11-28T11:11:24.391221+0000","last_peered":"2023-11-28T11:11:24.391221+0000","last_clean":"2023-11-28T11:11:24.391221+0000","last_became_active":"2023-11-28T11:11:24.005389+0000","last_became_peered":"2023-11-28T11:11:24.005389+0000","last_unstale":"2023-11-28T11:11:24.391221+0000","last_undegraded":"2023-11-28T11:11:24.391221+0000","last_fullsized":"2023-11-28T11:11:24.391221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387870+0000","last_change":"2023-11-28T11:11:24.400103+0000","last_active":"2023-11-28T11:11:25.387870+0000","last_peered":"2023-11-28T11:11:25.387870+0000","last_clean":"2023-11-28T11:11:25.387870+0000","last_became_active":"2023-11-28T11:11:24.399792+0000","last_became_peered":"2023-11-28T11:11:24.399792+0000","last_unstale":"2023-11-28T11:11:25.387870+0000","last_undegraded":"2023-11-28T11:11:25.387870+0000","last_fullsized":"2023-11-28T11:11:25.387870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:52.013060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390717+0000","last_change":"2023-11-28T11:11:24.006019+0000","last_active":"2023-11-28T11:11:24.390717+0000","last_peered":"2023-11-28T11:11:24.390717+0000","last_clean":"2023-11-28T11:11:24.390717+0000","last_became_active":"2023-11-28T11:11:24.005619+0000","last_became_peered":"2023-11-28T11:11:24.005619+0000","last_unstale":"2023-11-28T11:11:24.390717+0000","last_undegraded":"2023-11-28T11:11:24.390717+0000","last_fullsized":"2023-11-28T11:11:24.390717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:26.554018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388119+0000","last_change":"2023-11-28T11:11:24.410637+0000","last_active":"2023-11-28T11:11:25.388119+0000","last_peered":"2023-11-28T11:11:25.388119+0000","last_clean":"2023-11-28T11:11:25.388119+0000","last_became_active":"2023-11-28T11:11:24.410268+0000","last_became_peered":"2023-11-28T11:11:24.410268+0000","last_unstale":"2023-11-28T11:11:25.388119+0000","last_undegraded":"2023-11-28T11:11:25.388119+0000","last_fullsized":"2023-11-28T11:11:25.388119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:27.557596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401296+0000","last_change":"2023-11-28T11:11:24.001430+0000","last_active":"2023-11-28T11:11:24.401296+0000","last_peered":"2023-11-28T11:11:24.401296+0000","last_clean":"2023-11-28T11:11:24.401296+0000","last_became_active":"2023-11-28T11:11:24.001139+0000","last_became_peered":"2023-11-28T11:11:24.001139+0000","last_unstale":"2023-11-28T11:11:24.401296+0000","last_undegraded":"2023-11-28T11:11:24.401296+0000","last_fullsized":"2023-11-28T11:11:24.401296+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:27.874016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201623+0000","last_change":"2023-11-28T11:11:24.414682+0000","last_active":"2023-11-28T11:11:27.201623+0000","last_peered":"2023-11-28T11:11:27.201623+0000","last_clean":"2023-11-28T11:11:27.201623+0000","last_became_active":"2023-11-28T11:11:24.414182+0000","last_became_peered":"2023-11-28T11:11:24.414182+0000","last_unstale":"2023-11-28T11:11:27.201623+0000","last_undegraded":"2023-11-28T11:11:27.201623+0000","last_fullsized":"2023-11-28T11:11:27.201623+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:32.718969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.403024+0000","last_change":"2023-11-28T11:11:24.003771+0000","last_active":"2023-11-28T11:11:24.403024+0000","last_peered":"2023-11-28T11:11:24.403024+0000","last_clean":"2023-11-28T11:11:24.403024+0000","last_became_active":"2023-11-28T11:11:24.003517+0000","last_became_peered":"2023-11-28T11:11:24.003517+0000","last_unstale":"2023-11-28T11:11:24.403024+0000","last_undegraded":"2023-11-28T11:11:24.403024+0000","last_fullsized":"2023-11-28T11:11:24.403024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:09.661474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390761+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:11:24.390761+0000","last_peered":"2023-11-28T11:11:24.390761+0000","last_clean":"2023-11-28T11:11:24.390761+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:11:24.390761+0000","last_undegraded":"2023-11-28T11:11:24.390761+0000","last_fullsized":"2023-11-28T11:11:24.390761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201876+0000","last_change":"2023-11-28T11:11:24.398732+0000","last_active":"2023-11-28T11:11:27.201876+0000","last_peered":"2023-11-28T11:11:27.201876+0000","last_clean":"2023-11-28T11:11:27.201876+0000","last_became_active":"2023-11-28T11:11:24.398479+0000","last_became_peered":"2023-11-28T11:11:24.398479+0000","last_unstale":"2023-11-28T11:11:27.201876+0000","last_undegraded":"2023-11-28T11:11:27.201876+0000","last_fullsized":"2023-11-28T11:11:27.201876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:06.857506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415502+0000","last_change":"2023-11-28T11:11:24.004224+0000","last_active":"2023-11-28T11:11:24.415502+0000","last_peered":"2023-11-28T11:11:24.415502+0000","last_clean":"2023-11-28T11:11:24.415502+0000","last_became_active":"2023-11-28T11:11:24.003961+0000","last_became_peered":"2023-11-28T11:11:24.003961+0000","last_unstale":"2023-11-28T11:11:24.415502+0000","last_undegraded":"2023-11-28T11:11:24.415502+0000","last_fullsized":"2023-11-28T11:11:24.415502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.118149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.386113+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:11:24.386113+0000","last_peered":"2023-11-28T11:11:24.386113+0000","last_clean":"2023-11-28T11:11:24.386113+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:11:24.386113+0000","last_undegraded":"2023-11-28T11:11:24.386113+0000","last_fullsized":"2023-11-28T11:11:24.386113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200077+0000","last_change":"2023-11-28T11:11:24.407844+0000","last_active":"2023-11-28T11:11:27.200077+0000","last_peered":"2023-11-28T11:11:27.200077+0000","last_clean":"2023-11-28T11:11:27.200077+0000","last_became_active":"2023-11-28T11:11:24.407582+0000","last_became_peered":"2023-11-28T11:11:24.407582+0000","last_unstale":"2023-11-28T11:11:27.200077+0000","last_undegraded":"2023-11-28T11:11:27.200077+0000","last_fullsized":"2023-11-28T11:11:27.200077+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:51.338035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.392372+0000","last_change":"2023-11-28T11:11:24.001269+0000","last_active":"2023-11-28T11:11:24.392372+0000","last_peered":"2023-11-28T11:11:24.392372+0000","last_clean":"2023-11-28T11:11:24.392372+0000","last_became_active":"2023-11-28T11:11:24.000982+0000","last_became_peered":"2023-11-28T11:11:24.000982+0000","last_unstale":"2023-11-28T11:11:24.392372+0000","last_undegraded":"2023-11-28T11:11:24.392372+0000","last_fullsized":"2023-11-28T11:11:24.392372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:42.594460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390975+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:11:24.390975+0000","last_peered":"2023-11-28T11:11:24.390975+0000","last_clean":"2023-11-28T11:11:24.390975+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:11:24.390975+0000","last_undegraded":"2023-11-28T11:11:24.390975+0000","last_fullsized":"2023-11-28T11:11:24.390975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":197,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.393541+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:11:24.393541+0000","last_peered":"2023-11-28T11:11:24.393541+0000","last_clean":"2023-11-28T11:11:24.393541+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:11:24.393541+0000","last_undegraded":"2023-11-28T11:11:24.393541+0000","last_fullsized":"2023-11-28T11:11:24.393541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200044+0000","last_change":"2023-11-28T11:11:24.410140+0000","last_active":"2023-11-28T11:11:27.200044+0000","last_peered":"2023-11-28T11:11:27.200044+0000","last_clean":"2023-11-28T11:11:27.200044+0000","last_became_active":"2023-11-28T11:11:24.409866+0000","last_became_peered":"2023-11-28T11:11:24.409866+0000","last_unstale":"2023-11-28T11:11:27.200044+0000","last_undegraded":"2023-11-28T11:11:27.200044+0000","last_fullsized":"2023-11-28T11:11:27.200044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:54:18.633764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.386399+0000","last_change":"2023-11-28T11:11:24.000762+0000","last_active":"2023-11-28T11:11:24.386399+0000","last_peered":"2023-11-28T11:11:24.386399+0000","last_clean":"2023-11-28T11:11:24.386399+0000","last_became_active":"2023-11-28T11:11:24.000358+0000","last_became_peered":"2023-11-28T11:11:24.000358+0000","last_unstale":"2023-11-28T11:11:24.386399+0000","last_undegraded":"2023-11-28T11:11:24.386399+0000","last_fullsized":"2023-11-28T11:11:24.386399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:01.938679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.392714+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:11:24.392714+0000","last_peered":"2023-11-28T11:11:24.392714+0000","last_clean":"2023-11-28T11:11:24.392714+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:11:24.392714+0000","last_undegraded":"2023-11-28T11:11:24.392714+0000","last_fullsized":"2023-11-28T11:11:24.392714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200113+0000","last_change":"2023-11-28T11:11:24.410102+0000","last_active":"2023-11-28T11:11:27.200113+0000","last_peered":"2023-11-28T11:11:27.200113+0000","last_clean":"2023-11-28T11:11:27.200113+0000","last_became_active":"2023-11-28T11:11:24.409848+0000","last_became_peered":"2023-11-28T11:11:24.409848+0000","last_unstale":"2023-11-28T11:11:27.200113+0000","last_undegraded":"2023-11-28T11:11:27.200113+0000","last_fullsized":"2023-11-28T11:11:27.200113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:35.557184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.413788+0000","last_change":"2023-11-28T11:11:24.002609+0000","last_active":"2023-11-28T11:11:24.413788+0000","last_peered":"2023-11-28T11:11:24.413788+0000","last_clean":"2023-11-28T11:11:24.413788+0000","last_became_active":"2023-11-28T11:11:24.001692+0000","last_became_peered":"2023-11-28T11:11:24.001692+0000","last_unstale":"2023-11-28T11:11:24.413788+0000","last_undegraded":"2023-11-28T11:11:24.413788+0000","last_fullsized":"2023-11-28T11:11:24.413788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:25.947439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.393429+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:11:24.393429+0000","last_peered":"2023-11-28T11:11:24.393429+0000","last_clean":"2023-11-28T11:11:24.393429+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:11:24.393429+0000","last_undegraded":"2023-11-28T11:11:24.393429+0000","last_fullsized":"2023-11-28T11:11:24.393429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200119+0000","last_change":"2023-11-28T11:11:24.397994+0000","last_active":"2023-11-28T11:11:27.200119+0000","last_peered":"2023-11-28T11:11:27.200119+0000","last_clean":"2023-11-28T11:11:27.200119+0000","last_became_active":"2023-11-28T11:11:24.397717+0000","last_became_peered":"2023-11-28T11:11:24.397717+0000","last_unstale":"2023-11-28T11:11:27.200119+0000","last_undegraded":"2023-11-28T11:11:27.200119+0000","last_fullsized":"2023-11-28T11:11:27.200119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:03.452648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401837+0000","last_change":"2023-11-28T11:11:24.001177+0000","last_active":"2023-11-28T11:11:24.401837+0000","last_peered":"2023-11-28T11:11:24.401837+0000","last_clean":"2023-11-28T11:11:24.401837+0000","last_became_active":"2023-11-28T11:11:23.999347+0000","last_became_peered":"2023-11-28T11:11:23.999347+0000","last_unstale":"2023-11-28T11:11:24.401837+0000","last_undegraded":"2023-11-28T11:11:24.401837+0000","last_fullsized":"2023-11-28T11:11:24.401837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.385162+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:11:24.385162+0000","last_peered":"2023-11-28T11:11:24.385162+0000","last_clean":"2023-11-28T11:11:24.385162+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:11:24.385162+0000","last_undegraded":"2023-11-28T11:11:24.385162+0000","last_fullsized":"2023-11-28T11:11:24.385162+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388597+0000","last_change":"2023-11-28T11:11:24.397845+0000","last_active":"2023-11-28T11:11:25.388597+0000","last_peered":"2023-11-28T11:11:25.388597+0000","last_clean":"2023-11-28T11:11:25.388597+0000","last_became_active":"2023-11-28T11:11:24.397583+0000","last_became_peered":"2023-11-28T11:11:24.397583+0000","last_unstale":"2023-11-28T11:11:25.388597+0000","last_undegraded":"2023-11-28T11:11:25.388597+0000","last_fullsized":"2023-11-28T11:11:25.388597+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:38:21.477706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415297+0000","last_change":"2023-11-28T11:11:24.008615+0000","last_active":"2023-11-28T11:11:24.415297+0000","last_peered":"2023-11-28T11:11:24.415297+0000","last_clean":"2023-11-28T11:11:24.415297+0000","last_became_active":"2023-11-28T11:11:24.008323+0000","last_became_peered":"2023-11-28T11:11:24.008323+0000","last_unstale":"2023-11-28T11:11:24.415297+0000","last_undegraded":"2023-11-28T11:11:24.415297+0000","last_fullsized":"2023-11-28T11:11:24.415297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.402251+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:11:24.402251+0000","last_peered":"2023-11-28T11:11:24.402251+0000","last_clean":"2023-11-28T11:11:24.402251+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:11:24.402251+0000","last_undegraded":"2023-11-28T11:11:24.402251+0000","last_fullsized":"2023-11-28T11:11:24.402251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388219+0000","last_change":"2023-11-28T11:11:24.398453+0000","last_active":"2023-11-28T11:11:25.388219+0000","last_peered":"2023-11-28T11:11:25.388219+0000","last_clean":"2023-11-28T11:11:25.388219+0000","last_became_active":"2023-11-28T11:11:24.398173+0000","last_became_peered":"2023-11-28T11:11:24.398173+0000","last_unstale":"2023-11-28T11:11:25.388219+0000","last_undegraded":"2023-11-28T11:11:25.388219+0000","last_fullsized":"2023-11-28T11:11:25.388219+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:38:21.477706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.393481+0000","last_change":"2023-11-28T11:11:24.008610+0000","last_active":"2023-11-28T11:11:24.393481+0000","last_peered":"2023-11-28T11:11:24.393481+0000","last_clean":"2023-11-28T11:11:24.393481+0000","last_became_active":"2023-11-28T11:11:24.008302+0000","last_became_peered":"2023-11-28T11:11:24.008302+0000","last_unstale":"2023-11-28T11:11:24.393481+0000","last_undegraded":"2023-11-28T11:11:24.393481+0000","last_fullsized":"2023-11-28T11:11:24.393481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:44.971764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.402732+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:11:24.402732+0000","last_peered":"2023-11-28T11:11:24.402732+0000","last_clean":"2023-11-28T11:11:24.402732+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:11:24.402732+0000","last_undegraded":"2023-11-28T11:11:24.402732+0000","last_fullsized":"2023-11-28T11:11:24.402732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10748,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361092,"statfs":{"total":96636764160,"available":96625758208,"internally_reserved":0,"allocated":1363968,"data_stored":947747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.621},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9588,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362252,"statfs":{"total":96636764160,"available":96626946048,"internally_reserved":0,"allocated":765952,"data_stored":355510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":1.544,"15min":1.544},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.337,"15min":1.337},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":1.665,"15min":1.665},"last":0.44}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71399999999999997,"5min":1.762,"15min":1.762},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.629},{"interface":"front","average":{"1min":0.505,"5min":0.60599999999999998,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72899999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71799999999999997,"5min":1.002,"15min":1.002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70399999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.877},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.76700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10696,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361144,"statfs":{"total":96636764160,"available":96625811456,"internally_reserved":0,"allocated":1376256,"data_stored":949295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38,"15min":0.39000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58699999999999997,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.504,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.28899999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34499999999999997,"15min":0.34899999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.57399999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.252},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40999999999999998,"15min":0.41899999999999998},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.27400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.48499999999999999,"5min":1.363,"15min":1.363},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73799999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.752,"5min":1.54,"15min":1.54},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":1.48,"15min":1.48},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72899999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":1.004,"15min":1.004},"last":0.68000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9772,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362068,"statfs":{"total":96636764160,"available":96626757632,"internally_reserved":0,"allocated":757760,"data_stored":355418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47999999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.377},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32500000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.40699999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40100000000000002},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58899999999999997,"5min":0.871,"15min":0.871},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72999999999999998,"5min":0.872,"15min":0.872},"last":0.76700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.626}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9596,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362244,"statfs":{"total":96636764160,"available":96626937856,"internally_reserved":0,"allocated":774144,"data_stored":358327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.29599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.375}]},{"osd":3,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40400000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78000000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75800000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80900000000000005,"5min":1.024,"15min":1.024},"last":0.42699999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.436},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83499999999999996,"5min":1.05,"15min":1.05},"last":0.45200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9784,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362056,"statfs":{"total":96636764160,"available":96626745344,"internally_reserved":0,"allocated":770048,"data_stored":356891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.499,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.31},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35799999999999998,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":1.054,"15min":1.054},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53400000000000003,"5min":1.147,"15min":1.147},"last":0.36699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.58699999999999997,"15min":0.622},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":5.133,"15min":5.133},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68100000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.435},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.44900000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72599999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.194,"15min":1.194},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52300000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":1.077,"15min":1.077},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.48499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9844,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361996,"statfs":{"total":96636764160,"available":96626683904,"internally_reserved":0,"allocated":765952,"data_stored":356849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.44},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40100000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.498}]},{"osd":2,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67400000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.747,"15min":0.747},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72499999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.58799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9588,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362252,"statfs":{"total":96636764160,"available":96626946048,"internally_reserved":0,"allocated":765952,"data_stored":356869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.81299999999999994,"15min":0.86099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.497},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84799999999999998,"15min":0.90000000000000002},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70999999999999996,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.377}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":6.601,"15min":6.601},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.629}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:29.975 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-28T11:11:29.974+0000 7f20ad068700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:11:32.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:33.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.106+0000 7fd7fa894700 1 Processor -- start 2023-11-28T11:11:33.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.106+0000 7fd7fa894700 1 -- start start 2023-11-28T11:11:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7fa894700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f415d0e0 0x7fd7f40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7fa894700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f40b1fb0 0x7fd7f40cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7fa894700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f40cbfb0 con 0x7fd7f415cd10 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7fa894700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f406cee0 con 0x7fd7f40b1fb0 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7fa894700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40cb710 con 0x7fd7f415d0e0 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7f8e31700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f40b1fb0 0x7fd7f40cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7f3fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd7f415cd10 0x7fd7f40b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50656/0 (socket says 172.21.15.83:50656) 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7f8e31700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f40b1fb0 0x7fd7f40cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52800/0 (socket says 172.21.15.83:52800) 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.107+0000 7fd7f3fff700 1 -- 172.21.15.83:0/4186575891 learned_addr learned my addr 172.21.15.83:0/4186575891 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f37fe700 1 --2- 172.21.15.83:0/4186575891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f415d0e0 0x7fd7f40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f8e31700 1 -- 172.21.15.83:0/4186575891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f415d0e0 msgr2=0x7fd7f40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f8e31700 1 --2- 172.21.15.83:0/4186575891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f415d0e0 0x7fd7f40ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f8e31700 1 -- 172.21.15.83:0/4186575891 >> v1:172.21.15.83:6789/0 conn(0x7fd7f415cd10 legacy=0x7fd7f40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f8e31700 1 -- 172.21.15.83:0/4186575891 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40cb4e0 con 0x7fd7f40b1fb0 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/4186575891 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976503162 0 0) 0x7fd7f40cbfb0 con 0x7fd7f415cd10 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f8e31700 1 --2- 172.21.15.83:0/4186575891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f40b1fb0 0x7fd7f40cafa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009d90 tx=0x7fd7ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8709b28502f836 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/4186575891 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00e070 con 0x7fd7f40b1fb0 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/4186575891 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec00bbd0 con 0x7fd7f40b1fb0 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.108+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/4186575891 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec012430 con 0x7fd7f40b1fb0 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 -- 172.21.15.83:0/4186575891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f40b1fb0 msgr2=0x7fd7f40cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 --2- 172.21.15.83:0/4186575891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f40b1fb0 0x7fd7f40cafa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009d90 tx=0x7fd7ec00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 -- 172.21.15.83:0/4186575891 shutdown_connections 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 --2- 172.21.15.83:0/4186575891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f40b1fb0 0x7fd7f40cafa0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 --2- 172.21.15.83:0/4186575891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f415d0e0 0x7fd7f40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 -- 172.21.15.83:0/4186575891 >> 172.21.15.83:0/4186575891 conn(0x7fd7f406e9a0 msgr2=0x7fd7f40cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 -- 172.21.15.83:0/4186575891 shutdown_connections 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.109+0000 7fd7fa894700 1 -- 172.21.15.83:0/4186575891 wait complete. 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.110+0000 7fd7fa894700 1 Processor -- start 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.110+0000 7fd7fa894700 1 -- start start 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7fa894700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f40b1fb0 0x7fd7f40c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7fa894700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f415cd10 0x7fd7f40c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7fa894700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 0x7fd7f40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7fa894700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40c6c30 con 0x7fd7f415d0e0 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7fa894700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40c9500 con 0x7fd7f415cd10 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7fa894700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40c6110 con 0x7fd7f40b1fb0 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 0x7fd7f40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 0x7fd7f40c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52806/0 (socket says 172.21.15.83:52806) 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 -- 172.21.15.83:0/1602446599 learned_addr learned my addr 172.21.15.83:0/1602446599 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f37fe700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f415cd10 0x7fd7f40c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f3fff700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f40b1fb0 0x7fd7f40c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 -- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f40b1fb0 msgr2=0x7fd7f40c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f40b1fb0 0x7fd7f40c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 -- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f415cd10 msgr2=0x7fd7f40c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f415cd10 0x7fd7f40c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 -- 172.21.15.83:0/1602446599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec009a70 con 0x7fd7f415d0e0 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f37fe700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f415cd10 0x7fd7f40c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.111+0000 7fd7f8e31700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 0x7fd7f40c84e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec013000 tx=0x7fd7ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.112+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec003ea0 con 0x7fd7f415d0e0 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.112+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec020070 con 0x7fd7f415d0e0 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.112+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f40cbcd0 con 0x7fd7f415d0e0 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.112+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec004280 con 0x7fd7f415d0e0 2023-11-28T11:11:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.112+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f40cb9f0 con 0x7fd7f415d0e0 2023-11-28T11:11:33.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.112+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f406b9b0 con 0x7fd7f415d0e0 2023-11-28T11:11:33.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.113+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7ec0043e0 con 0x7fd7f415d0e0 2023-11-28T11:11:33.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.113+0000 7fd7f0ff9700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7e00682f0 0x7fd7e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.114+0000 7fd7f3fff700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7e00682f0 0x7fd7e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.114+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd7ec022070 con 0x7fd7f415d0e0 2023-11-28T11:11:33.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.114+0000 7fd7f3fff700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7e00682f0 0x7fd7e006a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8000f60 tx=0x7fd7e8007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.118+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7ec054a80 con 0x7fd7f415d0e0 2023-11-28T11:11:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.296+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7f4074320 con 0x7fd7e00682f0 2023-11-28T11:11:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.304+0000 7fd7f0ff9700 1 -- 172.21.15.83:0/1602446599 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311604 (secure 0 0 0) 0x7fd7f4074320 con 0x7fd7e00682f0 2023-11-28T11:11:33.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7e00682f0 msgr2=0x7fd7e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7e00682f0 0x7fd7e006a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8000f60 tx=0x7fd7e8007be0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 msgr2=0x7fd7f40c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 0x7fd7f40c84e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec013000 tx=0x7fd7ec004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 shutdown_connections 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd7e00682f0 0x7fd7e006a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f415d0e0 0x7fd7f40c84e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7f40b1fb0 0x7fd7f40c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 --2- 172.21.15.83:0/1602446599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7f415cd10 0x7fd7f40c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 >> 172.21.15.83:0/1602446599 conn(0x7fd7f406e9a0 msgr2=0x7fd7f40cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 shutdown_connections 2023-11-28T11:11:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.308+0000 7fd7fa894700 1 -- 172.21.15.83:0/1602446599 wait complete. 2023-11-28T11:11:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:11:33.321 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2023-11-28T11:11:31.606097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":79616,"kb_used_data":7168,"kb_used_omap":0,"kb_used_meta":72384,"kb_avail":754895104,"statfs":{"total":773094113280,"available":773012586496,"internally_reserved":0,"allocated":7340032,"data_stored":4036906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74121216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.620475"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201525+0000","last_change":"2023-11-28T11:11:24.411949+0000","last_active":"2023-11-28T11:11:27.201525+0000","last_peered":"2023-11-28T11:11:27.201525+0000","last_clean":"2023-11-28T11:11:27.201525+0000","last_became_active":"2023-11-28T11:11:24.409554+0000","last_became_peered":"2023-11-28T11:11:24.409554+0000","last_unstale":"2023-11-28T11:11:27.201525+0000","last_undegraded":"2023-11-28T11:11:27.201525+0000","last_fullsized":"2023-11-28T11:11:27.201525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:40.209892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388821+0000","last_change":"2023-11-28T11:11:24.409999+0000","last_active":"2023-11-28T11:11:25.388821+0000","last_peered":"2023-11-28T11:11:25.388821+0000","last_clean":"2023-11-28T11:11:25.388821+0000","last_became_active":"2023-11-28T11:11:24.409139+0000","last_became_peered":"2023-11-28T11:11:24.409139+0000","last_unstale":"2023-11-28T11:11:25.388821+0000","last_undegraded":"2023-11-28T11:11:25.388821+0000","last_fullsized":"2023-11-28T11:11:25.388821+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:01.031365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200218+0000","last_change":"2023-11-28T11:11:24.395340+0000","last_active":"2023-11-28T11:11:27.200218+0000","last_peered":"2023-11-28T11:11:27.200218+0000","last_clean":"2023-11-28T11:11:27.200218+0000","last_became_active":"2023-11-28T11:11:24.395094+0000","last_became_peered":"2023-11-28T11:11:24.395094+0000","last_unstale":"2023-11-28T11:11:27.200218+0000","last_undegraded":"2023-11-28T11:11:27.200218+0000","last_fullsized":"2023-11-28T11:11:27.200218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:24.943756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388485+0000","last_change":"2023-11-28T11:11:24.405522+0000","last_active":"2023-11-28T11:11:25.388485+0000","last_peered":"2023-11-28T11:11:25.388485+0000","last_clean":"2023-11-28T11:11:25.388485+0000","last_became_active":"2023-11-28T11:11:24.405145+0000","last_became_peered":"2023-11-28T11:11:24.405145+0000","last_unstale":"2023-11-28T11:11:25.388485+0000","last_undegraded":"2023-11-28T11:11:25.388485+0000","last_fullsized":"2023-11-28T11:11:25.388485+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:25.592467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387770+0000","last_change":"2023-11-28T11:11:24.403856+0000","last_active":"2023-11-28T11:11:25.387770+0000","last_peered":"2023-11-28T11:11:25.387770+0000","last_clean":"2023-11-28T11:11:25.387770+0000","last_became_active":"2023-11-28T11:11:24.403625+0000","last_became_peered":"2023-11-28T11:11:24.403625+0000","last_unstale":"2023-11-28T11:11:25.387770+0000","last_undegraded":"2023-11-28T11:11:25.387770+0000","last_fullsized":"2023-11-28T11:11:25.387770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:50.823320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388381+0000","last_change":"2023-11-28T11:11:24.407857+0000","last_active":"2023-11-28T11:11:25.388381+0000","last_peered":"2023-11-28T11:11:25.388381+0000","last_clean":"2023-11-28T11:11:25.388381+0000","last_became_active":"2023-11-28T11:11:24.405812+0000","last_became_peered":"2023-11-28T11:11:24.405812+0000","last_unstale":"2023-11-28T11:11:25.388381+0000","last_undegraded":"2023-11-28T11:11:25.388381+0000","last_fullsized":"2023-11-28T11:11:25.388381+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:18.535307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806226+0000","last_change":"2023-11-28T11:11:24.404376+0000","last_active":"2023-11-28T11:11:25.806226+0000","last_peered":"2023-11-28T11:11:25.806226+0000","last_clean":"2023-11-28T11:11:25.806226+0000","last_became_active":"2023-11-28T11:11:24.404099+0000","last_became_peered":"2023-11-28T11:11:24.404099+0000","last_unstale":"2023-11-28T11:11:25.806226+0000","last_undegraded":"2023-11-28T11:11:25.806226+0000","last_fullsized":"2023-11-28T11:11:25.806226+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:20.945993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388241+0000","last_change":"2023-11-28T11:11:24.409241+0000","last_active":"2023-11-28T11:11:25.388241+0000","last_peered":"2023-11-28T11:11:25.388241+0000","last_clean":"2023-11-28T11:11:25.388241+0000","last_became_active":"2023-11-28T11:11:24.408977+0000","last_became_peered":"2023-11-28T11:11:24.408977+0000","last_unstale":"2023-11-28T11:11:25.388241+0000","last_undegraded":"2023-11-28T11:11:25.388241+0000","last_fullsized":"2023-11-28T11:11:25.388241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:42.596380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388108+0000","last_change":"2023-11-28T11:11:24.396619+0000","last_active":"2023-11-28T11:11:25.388108+0000","last_peered":"2023-11-28T11:11:25.388108+0000","last_clean":"2023-11-28T11:11:25.388108+0000","last_became_active":"2023-11-28T11:11:24.396319+0000","last_became_peered":"2023-11-28T11:11:24.396319+0000","last_unstale":"2023-11-28T11:11:25.388108+0000","last_undegraded":"2023-11-28T11:11:25.388108+0000","last_fullsized":"2023-11-28T11:11:25.388108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:11.227361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201780+0000","last_change":"2023-11-28T11:11:24.396844+0000","last_active":"2023-11-28T11:11:27.201780+0000","last_peered":"2023-11-28T11:11:27.201780+0000","last_clean":"2023-11-28T11:11:27.201780+0000","last_became_active":"2023-11-28T11:11:24.396569+0000","last_became_peered":"2023-11-28T11:11:24.396569+0000","last_unstale":"2023-11-28T11:11:27.201780+0000","last_undegraded":"2023-11-28T11:11:27.201780+0000","last_fullsized":"2023-11-28T11:11:27.201780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:04.352002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201311+0000","last_change":"2023-11-28T11:11:24.402673+0000","last_active":"2023-11-28T11:11:27.201311+0000","last_peered":"2023-11-28T11:11:27.201311+0000","last_clean":"2023-11-28T11:11:27.201311+0000","last_became_active":"2023-11-28T11:11:24.402386+0000","last_became_peered":"2023-11-28T11:11:24.402386+0000","last_unstale":"2023-11-28T11:11:27.201311+0000","last_undegraded":"2023-11-28T11:11:27.201311+0000","last_fullsized":"2023-11-28T11:11:27.201311+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:06.857506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388552+0000","last_change":"2023-11-28T11:11:24.398769+0000","last_active":"2023-11-28T11:11:25.388552+0000","last_peered":"2023-11-28T11:11:25.388552+0000","last_clean":"2023-11-28T11:11:25.388552+0000","last_became_active":"2023-11-28T11:11:24.397605+0000","last_became_peered":"2023-11-28T11:11:24.397605+0000","last_unstale":"2023-11-28T11:11:25.388552+0000","last_undegraded":"2023-11-28T11:11:25.388552+0000","last_fullsized":"2023-11-28T11:11:25.388552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:53.772762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804225+0000","last_change":"2023-11-28T11:11:24.405277+0000","last_active":"2023-11-28T11:11:25.804225+0000","last_peered":"2023-11-28T11:11:25.804225+0000","last_clean":"2023-11-28T11:11:25.804225+0000","last_became_active":"2023-11-28T11:11:24.404991+0000","last_became_peered":"2023-11-28T11:11:24.404991+0000","last_unstale":"2023-11-28T11:11:25.804225+0000","last_undegraded":"2023-11-28T11:11:25.804225+0000","last_fullsized":"2023-11-28T11:11:25.804225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:37.550783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387700+0000","last_change":"2023-11-28T11:11:24.404417+0000","last_active":"2023-11-28T11:11:25.387700+0000","last_peered":"2023-11-28T11:11:25.387700+0000","last_clean":"2023-11-28T11:11:25.387700+0000","last_became_active":"2023-11-28T11:11:24.404162+0000","last_became_peered":"2023-11-28T11:11:24.404162+0000","last_unstale":"2023-11-28T11:11:25.387700+0000","last_undegraded":"2023-11-28T11:11:25.387700+0000","last_fullsized":"2023-11-28T11:11:25.387700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:35.756145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804034+0000","last_change":"2023-11-28T11:11:24.397418+0000","last_active":"2023-11-28T11:11:25.804034+0000","last_peered":"2023-11-28T11:11:25.804034+0000","last_clean":"2023-11-28T11:11:25.804034+0000","last_became_active":"2023-11-28T11:11:24.397166+0000","last_became_peered":"2023-11-28T11:11:24.397166+0000","last_unstale":"2023-11-28T11:11:25.804034+0000","last_undegraded":"2023-11-28T11:11:25.804034+0000","last_fullsized":"2023-11-28T11:11:25.804034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:26.608370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201161+0000","last_change":"2023-11-28T11:11:24.412720+0000","last_active":"2023-11-28T11:11:27.201161+0000","last_peered":"2023-11-28T11:11:27.201161+0000","last_clean":"2023-11-28T11:11:27.201161+0000","last_became_active":"2023-11-28T11:11:24.412463+0000","last_became_peered":"2023-11-28T11:11:24.412463+0000","last_unstale":"2023-11-28T11:11:27.201161+0000","last_undegraded":"2023-11-28T11:11:27.201161+0000","last_fullsized":"2023-11-28T11:11:27.201161+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:25.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388155+0000","last_change":"2023-11-28T11:11:24.394251+0000","last_active":"2023-11-28T11:11:25.388155+0000","last_peered":"2023-11-28T11:11:25.388155+0000","last_clean":"2023-11-28T11:11:25.388155+0000","last_became_active":"2023-11-28T11:11:24.394006+0000","last_became_peered":"2023-11-28T11:11:24.394006+0000","last_unstale":"2023-11-28T11:11:25.388155+0000","last_undegraded":"2023-11-28T11:11:25.388155+0000","last_fullsized":"2023-11-28T11:11:25.388155+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:55:57.581783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388499+0000","last_change":"2023-11-28T11:11:24.412428+0000","last_active":"2023-11-28T11:11:25.388499+0000","last_peered":"2023-11-28T11:11:25.388499+0000","last_clean":"2023-11-28T11:11:25.388499+0000","last_became_active":"2023-11-28T11:11:24.412152+0000","last_became_peered":"2023-11-28T11:11:24.412152+0000","last_unstale":"2023-11-28T11:11:25.388499+0000","last_undegraded":"2023-11-28T11:11:25.388499+0000","last_fullsized":"2023-11-28T11:11:25.388499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:45.975342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388703+0000","last_change":"2023-11-28T11:11:24.409100+0000","last_active":"2023-11-28T11:11:25.388703+0000","last_peered":"2023-11-28T11:11:25.388703+0000","last_clean":"2023-11-28T11:11:25.388703+0000","last_became_active":"2023-11-28T11:11:24.408794+0000","last_became_peered":"2023-11-28T11:11:24.408794+0000","last_unstale":"2023-11-28T11:11:25.388703+0000","last_undegraded":"2023-11-28T11:11:25.388703+0000","last_fullsized":"2023-11-28T11:11:25.388703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:52.656366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804213+0000","last_change":"2023-11-28T11:11:24.400243+0000","last_active":"2023-11-28T11:11:25.804213+0000","last_peered":"2023-11-28T11:11:25.804213+0000","last_clean":"2023-11-28T11:11:25.804213+0000","last_became_active":"2023-11-28T11:11:24.399969+0000","last_became_peered":"2023-11-28T11:11:24.399969+0000","last_unstale":"2023-11-28T11:11:25.804213+0000","last_undegraded":"2023-11-28T11:11:25.804213+0000","last_fullsized":"2023-11-28T11:11:25.804213+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:28.573445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201299+0000","last_change":"2023-11-28T11:11:24.397089+0000","last_active":"2023-11-28T11:11:27.201299+0000","last_peered":"2023-11-28T11:11:27.201299+0000","last_clean":"2023-11-28T11:11:27.201299+0000","last_became_active":"2023-11-28T11:11:24.396624+0000","last_became_peered":"2023-11-28T11:11:24.396624+0000","last_unstale":"2023-11-28T11:11:27.201299+0000","last_undegraded":"2023-11-28T11:11:27.201299+0000","last_fullsized":"2023-11-28T11:11:27.201299+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:13.808465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201482+0000","last_change":"2023-11-28T11:11:24.396407+0000","last_active":"2023-11-28T11:11:27.201482+0000","last_peered":"2023-11-28T11:11:27.201482+0000","last_clean":"2023-11-28T11:11:27.201482+0000","last_became_active":"2023-11-28T11:11:24.396140+0000","last_became_peered":"2023-11-28T11:11:24.396140+0000","last_unstale":"2023-11-28T11:11:27.201482+0000","last_undegraded":"2023-11-28T11:11:27.201482+0000","last_fullsized":"2023-11-28T11:11:27.201482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:20:22.499380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201131+0000","last_change":"2023-11-28T11:11:24.405692+0000","last_active":"2023-11-28T11:11:27.201131+0000","last_peered":"2023-11-28T11:11:27.201131+0000","last_clean":"2023-11-28T11:11:27.201131+0000","last_became_active":"2023-11-28T11:11:24.405422+0000","last_became_peered":"2023-11-28T11:11:24.405422+0000","last_unstale":"2023-11-28T11:11:27.201131+0000","last_undegraded":"2023-11-28T11:11:27.201131+0000","last_fullsized":"2023-11-28T11:11:27.201131+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:10.665053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201073+0000","last_change":"2023-11-28T11:11:24.413275+0000","last_active":"2023-11-28T11:11:27.201073+0000","last_peered":"2023-11-28T11:11:27.201073+0000","last_clean":"2023-11-28T11:11:27.201073+0000","last_became_active":"2023-11-28T11:11:24.413006+0000","last_became_peered":"2023-11-28T11:11:24.413006+0000","last_unstale":"2023-11-28T11:11:27.201073+0000","last_undegraded":"2023-11-28T11:11:27.201073+0000","last_fullsized":"2023-11-28T11:11:27.201073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:13.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201795+0000","last_change":"2023-11-28T11:11:24.410281+0000","last_active":"2023-11-28T11:11:27.201795+0000","last_peered":"2023-11-28T11:11:27.201795+0000","last_clean":"2023-11-28T11:11:27.201795+0000","last_became_active":"2023-11-28T11:11:24.410009+0000","last_became_peered":"2023-11-28T11:11:24.410009+0000","last_unstale":"2023-11-28T11:11:27.201795+0000","last_undegraded":"2023-11-28T11:11:27.201795+0000","last_fullsized":"2023-11-28T11:11:27.201795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:43.598038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806633+0000","last_change":"2023-11-28T11:11:24.412179+0000","last_active":"2023-11-28T11:11:25.806633+0000","last_peered":"2023-11-28T11:11:25.806633+0000","last_clean":"2023-11-28T11:11:25.806633+0000","last_became_active":"2023-11-28T11:11:24.411887+0000","last_became_peered":"2023-11-28T11:11:24.411887+0000","last_unstale":"2023-11-28T11:11:25.806633+0000","last_undegraded":"2023-11-28T11:11:25.806633+0000","last_fullsized":"2023-11-28T11:11:25.806633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:56.687482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200275+0000","last_change":"2023-11-28T11:11:24.403340+0000","last_active":"2023-11-28T11:11:27.200275+0000","last_peered":"2023-11-28T11:11:27.200275+0000","last_clean":"2023-11-28T11:11:27.200275+0000","last_became_active":"2023-11-28T11:11:24.402878+0000","last_became_peered":"2023-11-28T11:11:24.402878+0000","last_unstale":"2023-11-28T11:11:27.200275+0000","last_undegraded":"2023-11-28T11:11:27.200275+0000","last_fullsized":"2023-11-28T11:11:27.200275+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:52.798672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201535+0000","last_change":"2023-11-28T11:11:24.397777+0000","last_active":"2023-11-28T11:11:27.201535+0000","last_peered":"2023-11-28T11:11:27.201535+0000","last_clean":"2023-11-28T11:11:27.201535+0000","last_became_active":"2023-11-28T11:11:24.397497+0000","last_became_peered":"2023-11-28T11:11:24.397497+0000","last_unstale":"2023-11-28T11:11:27.201535+0000","last_undegraded":"2023-11-28T11:11:27.201535+0000","last_fullsized":"2023-11-28T11:11:27.201535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:40.209892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200078+0000","last_change":"2023-11-28T11:11:24.404569+0000","last_active":"2023-11-28T11:11:27.200078+0000","last_peered":"2023-11-28T11:11:27.200078+0000","last_clean":"2023-11-28T11:11:27.200078+0000","last_became_active":"2023-11-28T11:11:24.404324+0000","last_became_peered":"2023-11-28T11:11:24.404324+0000","last_unstale":"2023-11-28T11:11:27.200078+0000","last_undegraded":"2023-11-28T11:11:27.200078+0000","last_fullsized":"2023-11-28T11:11:27.200078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:00:00.415431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806369+0000","last_change":"2023-11-28T11:11:24.406440+0000","last_active":"2023-11-28T11:11:25.806369+0000","last_peered":"2023-11-28T11:11:25.806369+0000","last_clean":"2023-11-28T11:11:25.806369+0000","last_became_active":"2023-11-28T11:11:24.406153+0000","last_became_peered":"2023-11-28T11:11:24.406153+0000","last_unstale":"2023-11-28T11:11:25.806369+0000","last_undegraded":"2023-11-28T11:11:25.806369+0000","last_fullsized":"2023-11-28T11:11:25.806369+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:25.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806497+0000","last_change":"2023-11-28T11:11:24.389706+0000","last_active":"2023-11-28T11:11:25.806497+0000","last_peered":"2023-11-28T11:11:25.806497+0000","last_clean":"2023-11-28T11:11:25.806497+0000","last_became_active":"2023-11-28T11:11:24.389407+0000","last_became_peered":"2023-11-28T11:11:24.389407+0000","last_unstale":"2023-11-28T11:11:25.806497+0000","last_undegraded":"2023-11-28T11:11:25.806497+0000","last_fullsized":"2023-11-28T11:11:25.806497+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:27.262120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388214+0000","last_change":"2023-11-28T11:11:24.411153+0000","last_active":"2023-11-28T11:11:25.388214+0000","last_peered":"2023-11-28T11:11:25.388214+0000","last_clean":"2023-11-28T11:11:25.388214+0000","last_became_active":"2023-11-28T11:11:24.410909+0000","last_became_peered":"2023-11-28T11:11:24.410909+0000","last_unstale":"2023-11-28T11:11:25.388214+0000","last_undegraded":"2023-11-28T11:11:25.388214+0000","last_fullsized":"2023-11-28T11:11:25.388214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:04.527247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201543+0000","last_change":"2023-11-28T11:11:24.409531+0000","last_active":"2023-11-28T11:11:27.201543+0000","last_peered":"2023-11-28T11:11:27.201543+0000","last_clean":"2023-11-28T11:11:27.201543+0000","last_became_active":"2023-11-28T11:11:24.409240+0000","last_became_peered":"2023-11-28T11:11:24.409240+0000","last_unstale":"2023-11-28T11:11:27.201543+0000","last_undegraded":"2023-11-28T11:11:27.201543+0000","last_fullsized":"2023-11-28T11:11:27.201543+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:25.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.416496+0000","last_change":"2023-11-28T11:11:24.008585+0000","last_active":"2023-11-28T11:11:24.416496+0000","last_peered":"2023-11-28T11:11:24.416496+0000","last_clean":"2023-11-28T11:11:24.416496+0000","last_became_active":"2023-11-28T11:11:24.008309+0000","last_became_peered":"2023-11-28T11:11:24.008309+0000","last_unstale":"2023-11-28T11:11:24.416496+0000","last_undegraded":"2023-11-28T11:11:24.416496+0000","last_fullsized":"2023-11-28T11:11:24.416496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:51.009482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388212+0000","last_change":"2023-11-28T11:11:24.389701+0000","last_active":"2023-11-28T11:11:25.388212+0000","last_peered":"2023-11-28T11:11:25.388212+0000","last_clean":"2023-11-28T11:11:25.388212+0000","last_became_active":"2023-11-28T11:11:24.389461+0000","last_became_peered":"2023-11-28T11:11:24.389461+0000","last_unstale":"2023-11-28T11:11:25.388212+0000","last_undegraded":"2023-11-28T11:11:25.388212+0000","last_fullsized":"2023-11-28T11:11:25.388212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:58.121727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401532+0000","last_change":"2023-11-28T11:11:24.001889+0000","last_active":"2023-11-28T11:11:24.401532+0000","last_peered":"2023-11-28T11:11:24.401532+0000","last_clean":"2023-11-28T11:11:24.401532+0000","last_became_active":"2023-11-28T11:11:24.001614+0000","last_became_peered":"2023-11-28T11:11:24.001614+0000","last_unstale":"2023-11-28T11:11:24.401532+0000","last_undegraded":"2023-11-28T11:11:24.401532+0000","last_fullsized":"2023-11-28T11:11:24.401532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:03.348424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388601+0000","last_change":"2023-11-28T11:11:24.408287+0000","last_active":"2023-11-28T11:11:25.388601+0000","last_peered":"2023-11-28T11:11:25.388601+0000","last_clean":"2023-11-28T11:11:25.388601+0000","last_became_active":"2023-11-28T11:11:24.406043+0000","last_became_peered":"2023-11-28T11:11:24.406043+0000","last_unstale":"2023-11-28T11:11:25.388601+0000","last_undegraded":"2023-11-28T11:11:25.388601+0000","last_fullsized":"2023-11-28T11:11:25.388601+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:15.559457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.399451+0000","last_change":"2023-11-28T11:11:24.006533+0000","last_active":"2023-11-28T11:11:24.399451+0000","last_peered":"2023-11-28T11:11:24.399451+0000","last_clean":"2023-11-28T11:11:24.399451+0000","last_became_active":"2023-11-28T11:11:24.005330+0000","last_became_peered":"2023-11-28T11:11:24.005330+0000","last_unstale":"2023-11-28T11:11:24.399451+0000","last_undegraded":"2023-11-28T11:11:24.399451+0000","last_fullsized":"2023-11-28T11:11:24.399451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:12.037178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201398+0000","last_change":"2023-11-28T11:11:24.412461+0000","last_active":"2023-11-28T11:11:27.201398+0000","last_peered":"2023-11-28T11:11:27.201398+0000","last_clean":"2023-11-28T11:11:27.201398+0000","last_became_active":"2023-11-28T11:11:24.409781+0000","last_became_peered":"2023-11-28T11:11:24.409781+0000","last_unstale":"2023-11-28T11:11:27.201398+0000","last_undegraded":"2023-11-28T11:11:27.201398+0000","last_fullsized":"2023-11-28T11:11:27.201398+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:20.945993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.414283+0000","last_change":"2023-11-28T11:11:24.003944+0000","last_active":"2023-11-28T11:11:24.414283+0000","last_peered":"2023-11-28T11:11:24.414283+0000","last_clean":"2023-11-28T11:11:24.414283+0000","last_became_active":"2023-11-28T11:11:24.003537+0000","last_became_peered":"2023-11-28T11:11:24.003537+0000","last_unstale":"2023-11-28T11:11:24.414283+0000","last_undegraded":"2023-11-28T11:11:24.414283+0000","last_fullsized":"2023-11-28T11:11:24.414283+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:40.682923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806435+0000","last_change":"2023-11-28T11:11:24.400254+0000","last_active":"2023-11-28T11:11:25.806435+0000","last_peered":"2023-11-28T11:11:25.806435+0000","last_clean":"2023-11-28T11:11:25.806435+0000","last_became_active":"2023-11-28T11:11:24.395751+0000","last_became_peered":"2023-11-28T11:11:24.395751+0000","last_unstale":"2023-11-28T11:11:25.806435+0000","last_undegraded":"2023-11-28T11:11:25.806435+0000","last_fullsized":"2023-11-28T11:11:25.806435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:13.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.396818+0000","last_change":"2023-11-28T11:11:24.005782+0000","last_active":"2023-11-28T11:11:24.396818+0000","last_peered":"2023-11-28T11:11:24.396818+0000","last_clean":"2023-11-28T11:11:24.396818+0000","last_became_active":"2023-11-28T11:11:24.005497+0000","last_became_peered":"2023-11-28T11:11:24.005497+0000","last_unstale":"2023-11-28T11:11:24.396818+0000","last_undegraded":"2023-11-28T11:11:24.396818+0000","last_fullsized":"2023-11-28T11:11:24.396818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:08.956676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201405+0000","last_change":"2023-11-28T11:11:24.407156+0000","last_active":"2023-11-28T11:11:27.201405+0000","last_peered":"2023-11-28T11:11:27.201405+0000","last_clean":"2023-11-28T11:11:27.201405+0000","last_became_active":"2023-11-28T11:11:24.406819+0000","last_became_peered":"2023-11-28T11:11:24.406819+0000","last_unstale":"2023-11-28T11:11:27.201405+0000","last_undegraded":"2023-11-28T11:11:27.201405+0000","last_fullsized":"2023-11-28T11:11:27.201405+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:09.743552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.413902+0000","last_change":"2023-11-28T11:11:24.002284+0000","last_active":"2023-11-28T11:11:24.413902+0000","last_peered":"2023-11-28T11:11:24.413902+0000","last_clean":"2023-11-28T11:11:24.413902+0000","last_became_active":"2023-11-28T11:11:24.001981+0000","last_became_peered":"2023-11-28T11:11:24.001981+0000","last_unstale":"2023-11-28T11:11:24.413902+0000","last_undegraded":"2023-11-28T11:11:24.413902+0000","last_fullsized":"2023-11-28T11:11:24.413902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:49.819742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387264+0000","last_change":"2023-11-28T11:11:24.403553+0000","last_active":"2023-11-28T11:11:25.387264+0000","last_peered":"2023-11-28T11:11:25.387264+0000","last_clean":"2023-11-28T11:11:25.387264+0000","last_became_active":"2023-11-28T11:11:24.403273+0000","last_became_peered":"2023-11-28T11:11:24.403273+0000","last_unstale":"2023-11-28T11:11:25.387264+0000","last_undegraded":"2023-11-28T11:11:25.387264+0000","last_fullsized":"2023-11-28T11:11:25.387264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:04.090203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390263+0000","last_change":"2023-11-28T11:11:24.005594+0000","last_active":"2023-11-28T11:11:24.390263+0000","last_peered":"2023-11-28T11:11:24.390263+0000","last_clean":"2023-11-28T11:11:24.390263+0000","last_became_active":"2023-11-28T11:11:24.005372+0000","last_became_peered":"2023-11-28T11:11:24.005372+0000","last_unstale":"2023-11-28T11:11:24.390263+0000","last_undegraded":"2023-11-28T11:11:24.390263+0000","last_fullsized":"2023-11-28T11:11:24.390263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804078+0000","last_change":"2023-11-28T11:11:24.403292+0000","last_active":"2023-11-28T11:11:25.804078+0000","last_peered":"2023-11-28T11:11:25.804078+0000","last_clean":"2023-11-28T11:11:25.804078+0000","last_became_active":"2023-11-28T11:11:24.403029+0000","last_became_peered":"2023-11-28T11:11:24.403029+0000","last_unstale":"2023-11-28T11:11:25.804078+0000","last_undegraded":"2023-11-28T11:11:25.804078+0000","last_fullsized":"2023-11-28T11:11:25.804078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:34.701359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.383024+0000","last_change":"2023-11-28T11:11:23.998331+0000","last_active":"2023-11-28T11:11:24.383024+0000","last_peered":"2023-11-28T11:11:24.383024+0000","last_clean":"2023-11-28T11:11:24.383024+0000","last_became_active":"2023-11-28T11:11:23.997618+0000","last_became_peered":"2023-11-28T11:11:23.997618+0000","last_unstale":"2023-11-28T11:11:24.383024+0000","last_undegraded":"2023-11-28T11:11:24.383024+0000","last_fullsized":"2023-11-28T11:11:24.383024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:53.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806636+0000","last_change":"2023-11-28T11:11:24.401893+0000","last_active":"2023-11-28T11:11:25.806636+0000","last_peered":"2023-11-28T11:11:25.806636+0000","last_clean":"2023-11-28T11:11:25.806636+0000","last_became_active":"2023-11-28T11:11:24.401629+0000","last_became_peered":"2023-11-28T11:11:24.401629+0000","last_unstale":"2023-11-28T11:11:25.806636+0000","last_undegraded":"2023-11-28T11:11:25.806636+0000","last_fullsized":"2023-11-28T11:11:25.806636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:32.718969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390876+0000","last_change":"2023-11-28T11:11:24.006121+0000","last_active":"2023-11-28T11:11:24.390876+0000","last_peered":"2023-11-28T11:11:24.390876+0000","last_clean":"2023-11-28T11:11:24.390876+0000","last_became_active":"2023-11-28T11:11:24.005748+0000","last_became_peered":"2023-11-28T11:11:24.005748+0000","last_unstale":"2023-11-28T11:11:24.390876+0000","last_undegraded":"2023-11-28T11:11:24.390876+0000","last_fullsized":"2023-11-28T11:11:24.390876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:00.027787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201720+0000","last_change":"2023-11-28T11:11:24.413462+0000","last_active":"2023-11-28T11:11:27.201720+0000","last_peered":"2023-11-28T11:11:27.201720+0000","last_clean":"2023-11-28T11:11:27.201720+0000","last_became_active":"2023-11-28T11:11:24.411901+0000","last_became_peered":"2023-11-28T11:11:24.411901+0000","last_unstale":"2023-11-28T11:11:27.201720+0000","last_undegraded":"2023-11-28T11:11:27.201720+0000","last_fullsized":"2023-11-28T11:11:27.201720+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:15.559457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.402419+0000","last_change":"2023-11-28T11:11:24.002718+0000","last_active":"2023-11-28T11:11:24.402419+0000","last_peered":"2023-11-28T11:11:24.402419+0000","last_clean":"2023-11-28T11:11:24.402419+0000","last_became_active":"2023-11-28T11:11:24.002376+0000","last_became_peered":"2023-11-28T11:11:24.002376+0000","last_unstale":"2023-11-28T11:11:24.402419+0000","last_undegraded":"2023-11-28T11:11:24.402419+0000","last_fullsized":"2023-11-28T11:11:24.402419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388675+0000","last_change":"2023-11-28T11:11:24.402031+0000","last_active":"2023-11-28T11:11:25.388675+0000","last_peered":"2023-11-28T11:11:25.388675+0000","last_clean":"2023-11-28T11:11:25.388675+0000","last_became_active":"2023-11-28T11:11:24.401775+0000","last_became_peered":"2023-11-28T11:11:24.401775+0000","last_unstale":"2023-11-28T11:11:25.388675+0000","last_undegraded":"2023-11-28T11:11:25.388675+0000","last_fullsized":"2023-11-28T11:11:25.388675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:13.040756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.404264+0000","last_change":"2023-11-28T11:11:24.006140+0000","last_active":"2023-11-28T11:11:24.404264+0000","last_peered":"2023-11-28T11:11:24.404264+0000","last_clean":"2023-11-28T11:11:24.404264+0000","last_became_active":"2023-11-28T11:11:24.005908+0000","last_became_peered":"2023-11-28T11:11:24.005908+0000","last_unstale":"2023-11-28T11:11:24.404264+0000","last_undegraded":"2023-11-28T11:11:24.404264+0000","last_fullsized":"2023-11-28T11:11:24.404264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:06.471882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388114+0000","last_change":"2023-11-28T11:11:24.396415+0000","last_active":"2023-11-28T11:11:25.388114+0000","last_peered":"2023-11-28T11:11:25.388114+0000","last_clean":"2023-11-28T11:11:25.388114+0000","last_became_active":"2023-11-28T11:11:24.396147+0000","last_became_peered":"2023-11-28T11:11:24.396147+0000","last_unstale":"2023-11-28T11:11:25.388114+0000","last_undegraded":"2023-11-28T11:11:25.388114+0000","last_fullsized":"2023-11-28T11:11:25.388114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:04.662088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.409658+0000","last_change":"2023-11-28T11:11:23.997636+0000","last_active":"2023-11-28T11:11:24.409658+0000","last_peered":"2023-11-28T11:11:24.409658+0000","last_clean":"2023-11-28T11:11:24.409658+0000","last_became_active":"2023-11-28T11:11:23.997302+0000","last_became_peered":"2023-11-28T11:11:23.997302+0000","last_unstale":"2023-11-28T11:11:24.409658+0000","last_undegraded":"2023-11-28T11:11:24.409658+0000","last_fullsized":"2023-11-28T11:11:24.409658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:03.658510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201705+0000","last_change":"2023-11-28T11:11:24.404561+0000","last_active":"2023-11-28T11:11:27.201705+0000","last_peered":"2023-11-28T11:11:27.201705+0000","last_clean":"2023-11-28T11:11:27.201705+0000","last_became_active":"2023-11-28T11:11:24.404283+0000","last_became_peered":"2023-11-28T11:11:24.404283+0000","last_unstale":"2023-11-28T11:11:27.201705+0000","last_undegraded":"2023-11-28T11:11:27.201705+0000","last_fullsized":"2023-11-28T11:11:27.201705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:13.040756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.412464+0000","last_change":"2023-11-28T11:11:23.997769+0000","last_active":"2023-11-28T11:11:24.412464+0000","last_peered":"2023-11-28T11:11:24.412464+0000","last_clean":"2023-11-28T11:11:24.412464+0000","last_became_active":"2023-11-28T11:11:23.997513+0000","last_became_peered":"2023-11-28T11:11:23.997513+0000","last_unstale":"2023-11-28T11:11:24.412464+0000","last_undegraded":"2023-11-28T11:11:24.412464+0000","last_fullsized":"2023-11-28T11:11:24.412464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:51.735977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806350+0000","last_change":"2023-11-28T11:11:24.399918+0000","last_active":"2023-11-28T11:11:25.806350+0000","last_peered":"2023-11-28T11:11:25.806350+0000","last_clean":"2023-11-28T11:11:25.806350+0000","last_became_active":"2023-11-28T11:11:24.398006+0000","last_became_peered":"2023-11-28T11:11:24.398006+0000","last_unstale":"2023-11-28T11:11:25.806350+0000","last_undegraded":"2023-11-28T11:11:25.806350+0000","last_fullsized":"2023-11-28T11:11:25.806350+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:33.262768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401726+0000","last_change":"2023-11-28T11:11:24.002298+0000","last_active":"2023-11-28T11:11:24.401726+0000","last_peered":"2023-11-28T11:11:24.401726+0000","last_clean":"2023-11-28T11:11:24.401726+0000","last_became_active":"2023-11-28T11:11:24.002014+0000","last_became_peered":"2023-11-28T11:11:24.002014+0000","last_unstale":"2023-11-28T11:11:24.401726+0000","last_undegraded":"2023-11-28T11:11:24.401726+0000","last_fullsized":"2023-11-28T11:11:24.401726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:34.752567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.803999+0000","last_change":"2023-11-28T11:11:24.406640+0000","last_active":"2023-11-28T11:11:25.803999+0000","last_peered":"2023-11-28T11:11:25.803999+0000","last_clean":"2023-11-28T11:11:25.803999+0000","last_became_active":"2023-11-28T11:11:24.406387+0000","last_became_peered":"2023-11-28T11:11:24.406387+0000","last_unstale":"2023-11-28T11:11:25.803999+0000","last_undegraded":"2023-11-28T11:11:25.803999+0000","last_fullsized":"2023-11-28T11:11:25.803999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:48.019516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.385046+0000","last_change":"2023-11-28T11:11:24.001468+0000","last_active":"2023-11-28T11:11:24.385046+0000","last_peered":"2023-11-28T11:11:24.385046+0000","last_clean":"2023-11-28T11:11:24.385046+0000","last_became_active":"2023-11-28T11:11:24.001211+0000","last_became_peered":"2023-11-28T11:11:24.001211+0000","last_unstale":"2023-11-28T11:11:24.385046+0000","last_undegraded":"2023-11-28T11:11:24.385046+0000","last_fullsized":"2023-11-28T11:11:24.385046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:21.311957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387448+0000","last_change":"2023-11-28T11:11:24.395465+0000","last_active":"2023-11-28T11:11:25.387448+0000","last_peered":"2023-11-28T11:11:25.387448+0000","last_clean":"2023-11-28T11:11:25.387448+0000","last_became_active":"2023-11-28T11:11:24.395214+0000","last_became_peered":"2023-11-28T11:11:24.395214+0000","last_unstale":"2023-11-28T11:11:25.387448+0000","last_undegraded":"2023-11-28T11:11:25.387448+0000","last_fullsized":"2023-11-28T11:11:25.387448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:52.739555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415054+0000","last_change":"2023-11-28T11:11:24.008129+0000","last_active":"2023-11-28T11:11:24.415054+0000","last_peered":"2023-11-28T11:11:24.415054+0000","last_clean":"2023-11-28T11:11:24.415054+0000","last_became_active":"2023-11-28T11:11:24.007809+0000","last_became_peered":"2023-11-28T11:11:24.007809+0000","last_unstale":"2023-11-28T11:11:24.415054+0000","last_undegraded":"2023-11-28T11:11:24.415054+0000","last_fullsized":"2023-11-28T11:11:24.415054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:03.523669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.804260+0000","last_change":"2023-11-28T11:11:24.401697+0000","last_active":"2023-11-28T11:11:25.804260+0000","last_peered":"2023-11-28T11:11:25.804260+0000","last_clean":"2023-11-28T11:11:25.804260+0000","last_became_active":"2023-11-28T11:11:24.401374+0000","last_became_peered":"2023-11-28T11:11:24.401374+0000","last_unstale":"2023-11-28T11:11:25.804260+0000","last_undegraded":"2023-11-28T11:11:25.804260+0000","last_fullsized":"2023-11-28T11:11:25.804260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.621600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415600+0000","last_change":"2023-11-28T11:11:24.003527+0000","last_active":"2023-11-28T11:11:24.415600+0000","last_peered":"2023-11-28T11:11:24.415600+0000","last_clean":"2023-11-28T11:11:24.415600+0000","last_became_active":"2023-11-28T11:11:24.003238+0000","last_became_peered":"2023-11-28T11:11:24.003238+0000","last_unstale":"2023-11-28T11:11:24.415600+0000","last_undegraded":"2023-11-28T11:11:24.415600+0000","last_fullsized":"2023-11-28T11:11:24.415600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:20.035165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.806670+0000","last_change":"2023-11-28T11:11:24.400115+0000","last_active":"2023-11-28T11:11:25.806670+0000","last_peered":"2023-11-28T11:11:25.806670+0000","last_clean":"2023-11-28T11:11:25.806670+0000","last_became_active":"2023-11-28T11:11:24.395921+0000","last_became_peered":"2023-11-28T11:11:24.395921+0000","last_unstale":"2023-11-28T11:11:25.806670+0000","last_undegraded":"2023-11-28T11:11:25.806670+0000","last_fullsized":"2023-11-28T11:11:25.806670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:15.559457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.385231+0000","last_change":"2023-11-28T11:11:23.999779+0000","last_active":"2023-11-28T11:11:24.385231+0000","last_peered":"2023-11-28T11:11:24.385231+0000","last_clean":"2023-11-28T11:11:24.385231+0000","last_became_active":"2023-11-28T11:11:23.999526+0000","last_became_peered":"2023-11-28T11:11:23.999526+0000","last_unstale":"2023-11-28T11:11:24.385231+0000","last_undegraded":"2023-11-28T11:11:24.385231+0000","last_fullsized":"2023-11-28T11:11:24.385231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:28.037670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201286+0000","last_change":"2023-11-28T11:11:24.409840+0000","last_active":"2023-11-28T11:11:27.201286+0000","last_peered":"2023-11-28T11:11:27.201286+0000","last_clean":"2023-11-28T11:11:27.201286+0000","last_became_active":"2023-11-28T11:11:24.409407+0000","last_became_peered":"2023-11-28T11:11:24.409407+0000","last_unstale":"2023-11-28T11:11:27.201286+0000","last_undegraded":"2023-11-28T11:11:27.201286+0000","last_fullsized":"2023-11-28T11:11:27.201286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:20:22.499380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.384161+0000","last_change":"2023-11-28T11:11:24.001130+0000","last_active":"2023-11-28T11:11:24.384161+0000","last_peered":"2023-11-28T11:11:24.384161+0000","last_clean":"2023-11-28T11:11:24.384161+0000","last_became_active":"2023-11-28T11:11:24.000888+0000","last_became_peered":"2023-11-28T11:11:24.000888+0000","last_unstale":"2023-11-28T11:11:24.384161+0000","last_undegraded":"2023-11-28T11:11:24.384161+0000","last_fullsized":"2023-11-28T11:11:24.384161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:43:12.693701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201512+0000","last_change":"2023-11-28T11:11:24.413129+0000","last_active":"2023-11-28T11:11:27.201512+0000","last_peered":"2023-11-28T11:11:27.201512+0000","last_clean":"2023-11-28T11:11:27.201512+0000","last_became_active":"2023-11-28T11:11:24.412439+0000","last_became_peered":"2023-11-28T11:11:24.412439+0000","last_unstale":"2023-11-28T11:11:27.201512+0000","last_undegraded":"2023-11-28T11:11:27.201512+0000","last_fullsized":"2023-11-28T11:11:27.201512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:56.687482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.392498+0000","last_change":"2023-11-28T11:11:24.005822+0000","last_active":"2023-11-28T11:11:24.392498+0000","last_peered":"2023-11-28T11:11:24.392498+0000","last_clean":"2023-11-28T11:11:24.392498+0000","last_became_active":"2023-11-28T11:11:24.005558+0000","last_became_peered":"2023-11-28T11:11:24.005558+0000","last_unstale":"2023-11-28T11:11:24.392498+0000","last_undegraded":"2023-11-28T11:11:24.392498+0000","last_fullsized":"2023-11-28T11:11:24.392498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:10.349250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387781+0000","last_change":"2023-11-28T11:11:24.394292+0000","last_active":"2023-11-28T11:11:25.387781+0000","last_peered":"2023-11-28T11:11:25.387781+0000","last_clean":"2023-11-28T11:11:25.387781+0000","last_became_active":"2023-11-28T11:11:24.394024+0000","last_became_peered":"2023-11-28T11:11:24.394024+0000","last_unstale":"2023-11-28T11:11:25.387781+0000","last_undegraded":"2023-11-28T11:11:25.387781+0000","last_fullsized":"2023-11-28T11:11:25.387781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:21.038743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.383850+0000","last_change":"2023-11-28T11:11:23.996930+0000","last_active":"2023-11-28T11:11:24.383850+0000","last_peered":"2023-11-28T11:11:24.383850+0000","last_clean":"2023-11-28T11:11:24.383850+0000","last_became_active":"2023-11-28T11:11:23.996637+0000","last_became_peered":"2023-11-28T11:11:23.996637+0000","last_unstale":"2023-11-28T11:11:24.383850+0000","last_undegraded":"2023-11-28T11:11:24.383850+0000","last_fullsized":"2023-11-28T11:11:24.383850+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:36.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388500+0000","last_change":"2023-11-28T11:11:24.413366+0000","last_active":"2023-11-28T11:11:25.388500+0000","last_peered":"2023-11-28T11:11:25.388500+0000","last_clean":"2023-11-28T11:11:25.388500+0000","last_became_active":"2023-11-28T11:11:24.413074+0000","last_became_peered":"2023-11-28T11:11:24.413074+0000","last_unstale":"2023-11-28T11:11:25.388500+0000","last_undegraded":"2023-11-28T11:11:25.388500+0000","last_fullsized":"2023-11-28T11:11:25.388500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:10.665053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.391221+0000","last_change":"2023-11-28T11:11:24.005663+0000","last_active":"2023-11-28T11:11:24.391221+0000","last_peered":"2023-11-28T11:11:24.391221+0000","last_clean":"2023-11-28T11:11:24.391221+0000","last_became_active":"2023-11-28T11:11:24.005389+0000","last_became_peered":"2023-11-28T11:11:24.005389+0000","last_unstale":"2023-11-28T11:11:24.391221+0000","last_undegraded":"2023-11-28T11:11:24.391221+0000","last_fullsized":"2023-11-28T11:11:24.391221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.387870+0000","last_change":"2023-11-28T11:11:24.400103+0000","last_active":"2023-11-28T11:11:25.387870+0000","last_peered":"2023-11-28T11:11:25.387870+0000","last_clean":"2023-11-28T11:11:25.387870+0000","last_became_active":"2023-11-28T11:11:24.399792+0000","last_became_peered":"2023-11-28T11:11:24.399792+0000","last_unstale":"2023-11-28T11:11:25.387870+0000","last_undegraded":"2023-11-28T11:11:25.387870+0000","last_fullsized":"2023-11-28T11:11:25.387870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:52.013060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390717+0000","last_change":"2023-11-28T11:11:24.006019+0000","last_active":"2023-11-28T11:11:24.390717+0000","last_peered":"2023-11-28T11:11:24.390717+0000","last_clean":"2023-11-28T11:11:24.390717+0000","last_became_active":"2023-11-28T11:11:24.005619+0000","last_became_peered":"2023-11-28T11:11:24.005619+0000","last_unstale":"2023-11-28T11:11:24.390717+0000","last_undegraded":"2023-11-28T11:11:24.390717+0000","last_fullsized":"2023-11-28T11:11:24.390717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:26.554018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388119+0000","last_change":"2023-11-28T11:11:24.410637+0000","last_active":"2023-11-28T11:11:25.388119+0000","last_peered":"2023-11-28T11:11:25.388119+0000","last_clean":"2023-11-28T11:11:25.388119+0000","last_became_active":"2023-11-28T11:11:24.410268+0000","last_became_peered":"2023-11-28T11:11:24.410268+0000","last_unstale":"2023-11-28T11:11:25.388119+0000","last_undegraded":"2023-11-28T11:11:25.388119+0000","last_fullsized":"2023-11-28T11:11:25.388119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:27.557596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401296+0000","last_change":"2023-11-28T11:11:24.001430+0000","last_active":"2023-11-28T11:11:24.401296+0000","last_peered":"2023-11-28T11:11:24.401296+0000","last_clean":"2023-11-28T11:11:24.401296+0000","last_became_active":"2023-11-28T11:11:24.001139+0000","last_became_peered":"2023-11-28T11:11:24.001139+0000","last_unstale":"2023-11-28T11:11:24.401296+0000","last_undegraded":"2023-11-28T11:11:24.401296+0000","last_fullsized":"2023-11-28T11:11:24.401296+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:39:27.874016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201623+0000","last_change":"2023-11-28T11:11:24.414682+0000","last_active":"2023-11-28T11:11:27.201623+0000","last_peered":"2023-11-28T11:11:27.201623+0000","last_clean":"2023-11-28T11:11:27.201623+0000","last_became_active":"2023-11-28T11:11:24.414182+0000","last_became_peered":"2023-11-28T11:11:24.414182+0000","last_unstale":"2023-11-28T11:11:27.201623+0000","last_undegraded":"2023-11-28T11:11:27.201623+0000","last_fullsized":"2023-11-28T11:11:27.201623+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:32.718969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.403024+0000","last_change":"2023-11-28T11:11:24.003771+0000","last_active":"2023-11-28T11:11:24.403024+0000","last_peered":"2023-11-28T11:11:24.403024+0000","last_clean":"2023-11-28T11:11:24.403024+0000","last_became_active":"2023-11-28T11:11:24.003517+0000","last_became_peered":"2023-11-28T11:11:24.003517+0000","last_unstale":"2023-11-28T11:11:24.403024+0000","last_undegraded":"2023-11-28T11:11:24.403024+0000","last_fullsized":"2023-11-28T11:11:24.403024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:09.661474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390761+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:11:24.390761+0000","last_peered":"2023-11-28T11:11:24.390761+0000","last_clean":"2023-11-28T11:11:24.390761+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:11:24.390761+0000","last_undegraded":"2023-11-28T11:11:24.390761+0000","last_fullsized":"2023-11-28T11:11:24.390761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.201876+0000","last_change":"2023-11-28T11:11:24.398732+0000","last_active":"2023-11-28T11:11:27.201876+0000","last_peered":"2023-11-28T11:11:27.201876+0000","last_clean":"2023-11-28T11:11:27.201876+0000","last_became_active":"2023-11-28T11:11:24.398479+0000","last_became_peered":"2023-11-28T11:11:24.398479+0000","last_unstale":"2023-11-28T11:11:27.201876+0000","last_undegraded":"2023-11-28T11:11:27.201876+0000","last_fullsized":"2023-11-28T11:11:27.201876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:06.857506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415502+0000","last_change":"2023-11-28T11:11:24.004224+0000","last_active":"2023-11-28T11:11:24.415502+0000","last_peered":"2023-11-28T11:11:24.415502+0000","last_clean":"2023-11-28T11:11:24.415502+0000","last_became_active":"2023-11-28T11:11:24.003961+0000","last_became_peered":"2023-11-28T11:11:24.003961+0000","last_unstale":"2023-11-28T11:11:24.415502+0000","last_undegraded":"2023-11-28T11:11:24.415502+0000","last_fullsized":"2023-11-28T11:11:24.415502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.118149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.386113+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:11:24.386113+0000","last_peered":"2023-11-28T11:11:24.386113+0000","last_clean":"2023-11-28T11:11:24.386113+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:11:24.386113+0000","last_undegraded":"2023-11-28T11:11:24.386113+0000","last_fullsized":"2023-11-28T11:11:24.386113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200077+0000","last_change":"2023-11-28T11:11:24.407844+0000","last_active":"2023-11-28T11:11:27.200077+0000","last_peered":"2023-11-28T11:11:27.200077+0000","last_clean":"2023-11-28T11:11:27.200077+0000","last_became_active":"2023-11-28T11:11:24.407582+0000","last_became_peered":"2023-11-28T11:11:24.407582+0000","last_unstale":"2023-11-28T11:11:27.200077+0000","last_undegraded":"2023-11-28T11:11:27.200077+0000","last_fullsized":"2023-11-28T11:11:27.200077+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:51.338035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.392372+0000","last_change":"2023-11-28T11:11:24.001269+0000","last_active":"2023-11-28T11:11:24.392372+0000","last_peered":"2023-11-28T11:11:24.392372+0000","last_clean":"2023-11-28T11:11:24.392372+0000","last_became_active":"2023-11-28T11:11:24.000982+0000","last_became_peered":"2023-11-28T11:11:24.000982+0000","last_unstale":"2023-11-28T11:11:24.392372+0000","last_undegraded":"2023-11-28T11:11:24.392372+0000","last_fullsized":"2023-11-28T11:11:24.392372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:42.594460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.390975+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:11:24.390975+0000","last_peered":"2023-11-28T11:11:24.390975+0000","last_clean":"2023-11-28T11:11:24.390975+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:11:24.390975+0000","last_undegraded":"2023-11-28T11:11:24.390975+0000","last_fullsized":"2023-11-28T11:11:24.390975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":197,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.393541+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:11:24.393541+0000","last_peered":"2023-11-28T11:11:24.393541+0000","last_clean":"2023-11-28T11:11:24.393541+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:11:24.393541+0000","last_undegraded":"2023-11-28T11:11:24.393541+0000","last_fullsized":"2023-11-28T11:11:24.393541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200044+0000","last_change":"2023-11-28T11:11:24.410140+0000","last_active":"2023-11-28T11:11:27.200044+0000","last_peered":"2023-11-28T11:11:27.200044+0000","last_clean":"2023-11-28T11:11:27.200044+0000","last_became_active":"2023-11-28T11:11:24.409866+0000","last_became_peered":"2023-11-28T11:11:24.409866+0000","last_unstale":"2023-11-28T11:11:27.200044+0000","last_undegraded":"2023-11-28T11:11:27.200044+0000","last_fullsized":"2023-11-28T11:11:27.200044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:54:18.633764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.386399+0000","last_change":"2023-11-28T11:11:24.000762+0000","last_active":"2023-11-28T11:11:24.386399+0000","last_peered":"2023-11-28T11:11:24.386399+0000","last_clean":"2023-11-28T11:11:24.386399+0000","last_became_active":"2023-11-28T11:11:24.000358+0000","last_became_peered":"2023-11-28T11:11:24.000358+0000","last_unstale":"2023-11-28T11:11:24.386399+0000","last_undegraded":"2023-11-28T11:11:24.386399+0000","last_fullsized":"2023-11-28T11:11:24.386399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:01.938679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.392714+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:11:24.392714+0000","last_peered":"2023-11-28T11:11:24.392714+0000","last_clean":"2023-11-28T11:11:24.392714+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:11:24.392714+0000","last_undegraded":"2023-11-28T11:11:24.392714+0000","last_fullsized":"2023-11-28T11:11:24.392714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200113+0000","last_change":"2023-11-28T11:11:24.410102+0000","last_active":"2023-11-28T11:11:27.200113+0000","last_peered":"2023-11-28T11:11:27.200113+0000","last_clean":"2023-11-28T11:11:27.200113+0000","last_became_active":"2023-11-28T11:11:24.409848+0000","last_became_peered":"2023-11-28T11:11:24.409848+0000","last_unstale":"2023-11-28T11:11:27.200113+0000","last_undegraded":"2023-11-28T11:11:27.200113+0000","last_fullsized":"2023-11-28T11:11:27.200113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:35.557184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.413788+0000","last_change":"2023-11-28T11:11:24.002609+0000","last_active":"2023-11-28T11:11:24.413788+0000","last_peered":"2023-11-28T11:11:24.413788+0000","last_clean":"2023-11-28T11:11:24.413788+0000","last_became_active":"2023-11-28T11:11:24.001692+0000","last_became_peered":"2023-11-28T11:11:24.001692+0000","last_unstale":"2023-11-28T11:11:24.413788+0000","last_undegraded":"2023-11-28T11:11:24.413788+0000","last_fullsized":"2023-11-28T11:11:24.413788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:18:25.947439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.393429+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:11:24.393429+0000","last_peered":"2023-11-28T11:11:24.393429+0000","last_clean":"2023-11-28T11:11:24.393429+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:11:24.393429+0000","last_undegraded":"2023-11-28T11:11:24.393429+0000","last_fullsized":"2023-11-28T11:11:24.393429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:27.200119+0000","last_change":"2023-11-28T11:11:24.397994+0000","last_active":"2023-11-28T11:11:27.200119+0000","last_peered":"2023-11-28T11:11:27.200119+0000","last_clean":"2023-11-28T11:11:27.200119+0000","last_became_active":"2023-11-28T11:11:24.397717+0000","last_became_peered":"2023-11-28T11:11:24.397717+0000","last_unstale":"2023-11-28T11:11:27.200119+0000","last_undegraded":"2023-11-28T11:11:27.200119+0000","last_fullsized":"2023-11-28T11:11:27.200119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:03.452648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.401837+0000","last_change":"2023-11-28T11:11:24.001177+0000","last_active":"2023-11-28T11:11:24.401837+0000","last_peered":"2023-11-28T11:11:24.401837+0000","last_clean":"2023-11-28T11:11:24.401837+0000","last_became_active":"2023-11-28T11:11:23.999347+0000","last_became_peered":"2023-11-28T11:11:23.999347+0000","last_unstale":"2023-11-28T11:11:24.401837+0000","last_undegraded":"2023-11-28T11:11:24.401837+0000","last_fullsized":"2023-11-28T11:11:24.401837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:03.086625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.385162+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:11:24.385162+0000","last_peered":"2023-11-28T11:11:24.385162+0000","last_clean":"2023-11-28T11:11:24.385162+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:11:24.385162+0000","last_undegraded":"2023-11-28T11:11:24.385162+0000","last_fullsized":"2023-11-28T11:11:24.385162+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388597+0000","last_change":"2023-11-28T11:11:24.397845+0000","last_active":"2023-11-28T11:11:25.388597+0000","last_peered":"2023-11-28T11:11:25.388597+0000","last_clean":"2023-11-28T11:11:25.388597+0000","last_became_active":"2023-11-28T11:11:24.397583+0000","last_became_peered":"2023-11-28T11:11:24.397583+0000","last_unstale":"2023-11-28T11:11:25.388597+0000","last_undegraded":"2023-11-28T11:11:25.388597+0000","last_fullsized":"2023-11-28T11:11:25.388597+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:38:21.477706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.415297+0000","last_change":"2023-11-28T11:11:24.008615+0000","last_active":"2023-11-28T11:11:24.415297+0000","last_peered":"2023-11-28T11:11:24.415297+0000","last_clean":"2023-11-28T11:11:24.415297+0000","last_became_active":"2023-11-28T11:11:24.008323+0000","last_became_peered":"2023-11-28T11:11:24.008323+0000","last_unstale":"2023-11-28T11:11:24.415297+0000","last_undegraded":"2023-11-28T11:11:24.415297+0000","last_fullsized":"2023-11-28T11:11:24.415297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:16.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.402251+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:11:24.402251+0000","last_peered":"2023-11-28T11:11:24.402251+0000","last_clean":"2023-11-28T11:11:24.402251+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:11:24.402251+0000","last_undegraded":"2023-11-28T11:11:24.402251+0000","last_fullsized":"2023-11-28T11:11:24.402251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:11:25.388219+0000","last_change":"2023-11-28T11:11:24.398453+0000","last_active":"2023-11-28T11:11:25.388219+0000","last_peered":"2023-11-28T11:11:25.388219+0000","last_clean":"2023-11-28T11:11:25.388219+0000","last_became_active":"2023-11-28T11:11:24.398173+0000","last_became_peered":"2023-11-28T11:11:24.398173+0000","last_unstale":"2023-11-28T11:11:25.388219+0000","last_undegraded":"2023-11-28T11:11:25.388219+0000","last_fullsized":"2023-11-28T11:11:25.388219+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:23.983190+0000","last_clean_scrub_stamp":"2023-11-28T11:11:23.983190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:38:21.477706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.393481+0000","last_change":"2023-11-28T11:11:24.008610+0000","last_active":"2023-11-28T11:11:24.393481+0000","last_peered":"2023-11-28T11:11:24.393481+0000","last_clean":"2023-11-28T11:11:24.393481+0000","last_became_active":"2023-11-28T11:11:24.008302+0000","last_became_peered":"2023-11-28T11:11:24.008302+0000","last_unstale":"2023-11-28T11:11:24.393481+0000","last_undegraded":"2023-11-28T11:11:24.393481+0000","last_fullsized":"2023-11-28T11:11:24.393481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:11:22.979612+0000","last_clean_scrub_stamp":"2023-11-28T11:11:22.979612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:44.971764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:24.402732+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:11:24.402732+0000","last_peered":"2023-11-28T11:11:24.402732+0000","last_clean":"2023-11-28T11:11:24.402732+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:11:24.402732+0000","last_undegraded":"2023-11-28T11:11:24.402732+0000","last_fullsized":"2023-11-28T11:11:24.402732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10748,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361092,"statfs":{"total":96636764160,"available":96625758208,"internally_reserved":0,"allocated":1363968,"data_stored":947747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.621},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9588,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362252,"statfs":{"total":96636764160,"available":96626946048,"internally_reserved":0,"allocated":765952,"data_stored":355510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":1.544,"15min":1.544},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.337,"15min":1.337},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":1.665,"15min":1.665},"last":0.44}]},{"osd":3,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71399999999999997,"5min":1.762,"15min":1.762},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.629},{"interface":"front","average":{"1min":0.505,"5min":0.60599999999999998,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72899999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71799999999999997,"5min":1.002,"15min":1.002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70399999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:10:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.877},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.76700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10696,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361144,"statfs":{"total":96636764160,"available":96625811456,"internally_reserved":0,"allocated":1376256,"data_stored":949295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38,"15min":0.39000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58699999999999997,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.504,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.28899999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.34499999999999997,"15min":0.34899999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.57399999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.252},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40999999999999998,"15min":0.41899999999999998},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.27400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.48499999999999999,"5min":1.363,"15min":1.363},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73799999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73199999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.752,"5min":1.54,"15min":1.54},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":1.48,"15min":1.48},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72899999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":1.004,"15min":1.004},"last":0.68000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9772,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362068,"statfs":{"total":96636764160,"available":96626757632,"internally_reserved":0,"allocated":757760,"data_stored":355418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47999999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.377},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32500000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.40699999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40100000000000002},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58899999999999997,"5min":0.871,"15min":0.871},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79100000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72999999999999998,"5min":0.872,"15min":0.872},"last":0.76700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.626}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9596,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362244,"statfs":{"total":96636764160,"available":96626937856,"internally_reserved":0,"allocated":774144,"data_stored":358327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.29599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.375}]},{"osd":3,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.374,"15min":0.377},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40400000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78000000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75800000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80900000000000005,"5min":1.024,"15min":1.024},"last":0.42699999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.436},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83499999999999996,"5min":1.05,"15min":1.05},"last":0.45200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9784,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362056,"statfs":{"total":96636764160,"available":96626745344,"internally_reserved":0,"allocated":770048,"data_stored":356891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.499,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.31},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35799999999999998,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":1.054,"15min":1.054},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53400000000000003,"5min":1.147,"15min":1.147},"last":0.36699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.58699999999999997,"15min":0.622},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":5.133,"15min":5.133},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68100000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.435},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.44900000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72599999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.194,"15min":1.194},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52300000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78000000000000003,"5min":1.077,"15min":1.077},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.48499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9844,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361996,"statfs":{"total":96636764160,"available":96626683904,"internally_reserved":0,"allocated":765952,"data_stored":356849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.44},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40100000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.498}]},{"osd":2,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67400000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.747,"15min":0.747},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72499999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.58799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9588,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362252,"statfs":{"total":96636764160,"available":96626946048,"internally_reserved":0,"allocated":765952,"data_stored":356869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.81299999999999994,"15min":0.86099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.497},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84799999999999998,"15min":0.90000000000000002},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70999999999999996,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.377}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":6.601,"15min":6.601},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.629}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:33.327 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:11:33.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:11:33.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.488+0000 7fa0ae8c4700 1 Processor -- start 2023-11-28T11:11:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.488+0000 7fa0ae8c4700 1 -- start start 2023-11-28T11:11:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0ae8c4700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa0a80b64c0 0x7fa0a80b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0ae8c4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a80cb890 con 0x7fa0a80cf4c0 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0ae8c4700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a80cf960 con 0x7fa0a80b4290 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0ae8c4700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a80b6890 con 0x7fa0a80b64c0 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0a77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa0a80cf4c0 0x7fa0a80b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50672/0 (socket says 172.21.15.83:50672) 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0a77fe700 1 -- 172.21.15.83:0/46868894 learned_addr learned my addr 172.21.15.83:0/46868894 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173917422 0 0) 0x7fa0a80cb890 con 0x7fa0a80cf4c0 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0a7fff700 1 --2- 172.21.15.83:0/46868894 >> v2:172.21.15.90:3300/0 conn(0x7fa0a80b64c0 0x7fa0a80b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.489+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0900035b0 con 0x7fa0a80cf4c0 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756293921 0 0) 0x7fa0900035b0 con 0x7fa0a80cf4c0 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 >> v2:172.21.15.90:3300/0 conn(0x7fa0a80b64c0 msgr2=0x7fa0a80b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 --2- 172.21.15.83:0/46868894 >> v2:172.21.15.90:3300/0 conn(0x7fa0a80b64c0 0x7fa0a80b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 >> v1:172.21.15.90:6789/0 conn(0x7fa0a80b4290 legacy=0x7fa0a80cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a806fb10 con 0x7fa0a80cf4c0 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fa098001670 con 0x7fa0a80cf4c0 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 >> v1:172.21.15.83:6789/0 conn(0x7fa0a80cf4c0 legacy=0x7fa0a80b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa090003a00 0x7fa090003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa090004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa090005200 0x7fa09000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80b6890 con 0x7fa090004850 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09000c080 con 0x7fa090005200 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09000c240 con 0x7fa090003a00 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa090004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a7fff700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa090003a00 0x7fa090003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0ace61700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa090005200 0x7fa09000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 -- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa090003a00 msgr2=0x7fa090003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa090003a00 0x7fa090003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 -- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa090005200 msgr2=0x7fa09000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa090005200 0x7fa09000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0ace61700 1 -- 172.21.15.83:0/46868894 reap_dead start 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 -- 172.21.15.83:0/46868894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900035b0 con 0x7fa090004850 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa090004cc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c2a0 tx=0x7fa090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afa2e2d31f3b383 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098007c10 con 0x7fa090004850 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09800cdc0 con 0x7fa090004850 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.490+0000 7fa0a6ffd700 1 -- 172.21.15.83:0/46868894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098005ba0 con 0x7fa090004850 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/46868894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 msgr2=0x7fa090004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa090004cc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c2a0 tx=0x7fa090005880 comp rx=0 tx=0).stop 2023-11-28T11:11:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/46868894 shutdown_connections 2023-11-28T11:11:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/46868894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa090004cc0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/46868894 >> 172.21.15.83:0/46868894 conn(0x7fa0a80708a0 msgr2=0x7fa0a8070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/46868894 shutdown_connections 2023-11-28T11:11:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.491+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/46868894 wait complete. 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.492+0000 7fa0ae8c4700 1 Processor -- start 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.493+0000 7fa0ae8c4700 1 -- start start 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa0a81591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a80ce8e0 0x7fa0a80ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa0a80a7920 0x7fa0a80a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80cf290 con 0x7fa090004850 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80a8630 con 0x7fa0a80ce8e0 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80a8910 con 0x7fa0a80a7920 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa0a81591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa0a81591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52822/0 (socket says 172.21.15.83:52822) 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 -- 172.21.15.83:0/2091151401 learned_addr learned my addr 172.21.15.83:0/2091151401 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ace61700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa0a80a7920 0x7fa0a80a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a80ce8e0 0x7fa0a80ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 -- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa0a80a7920 msgr2=0x7fa0a80a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa0a80a7920 0x7fa0a80a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 -- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a80ce8e0 msgr2=0x7fa0a80ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a80ce8e0 0x7fa0a80ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 -- 172.21.15.83:0/2091151401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80cd2f0 con 0x7fa090004850 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a7fff700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa0a81591e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa09c011aa0 tx=0x7fa09c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00f9e0 con 0x7fa090004850 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00fb40 con 0x7fa090004850 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a80cd010 con 0x7fa090004850 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c014440 con 0x7fa090004850 2023-11-28T11:11:33.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.494+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a80cb520 con 0x7fa090004850 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.495+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a8074240 con 0x7fa090004850 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.496+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa09c01d400 con 0x7fa090004850 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.496+0000 7fa0a4ff9700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa094068320 0x7fa09406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.496+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa09c086bd0 con 0x7fa090004850 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.496+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa094068320 0x7fa09406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.497+0000 7fa0a77fe700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa094068320 0x7fa09406a7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa098001930 tx=0x7fa098007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.500+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa09c0533e0 con 0x7fa090004850 2023-11-28T11:11:33.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.758+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fa0a80b0f90 con 0x7fa090004850 2023-11-28T11:11:33.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.762+0000 7fa0a4ff9700 1 -- 172.21.15.83:0/2091151401 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa09c0220e0 con 0x7fa090004850 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa094068320 msgr2=0x7fa09406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa094068320 0x7fa09406a7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa098001930 tx=0x7fa098007000 comp rx=0 tx=0).stop 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 msgr2=0x7fa0a81591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa0a81591e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa09c011aa0 tx=0x7fa09c011e60 comp rx=0 tx=0).stop 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 shutdown_connections 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa0a80a7920 0x7fa0a80a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa094068320 0x7fa09406a7d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa090004850 0x7fa0a81591e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 --2- 172.21.15.83:0/2091151401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a80ce8e0 0x7fa0a80ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 >> 172.21.15.83:0/2091151401 conn(0x7fa0a80708a0 msgr2=0x7fa0a80b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:33.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 shutdown_connections 2023-11-28T11:11:33.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.764+0000 7fa0ae8c4700 1 -- 172.21.15.83:0/2091151401 wait complete. 2023-11-28T11:11:33.767 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-28T11:11:33.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.900+0000 7f1cb0e60700 1 Processor -- start 2023-11-28T11:11:33.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.900+0000 7f1cb0e60700 1 -- start start 2023-11-28T11:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1cb0e60700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 0x7f1cac15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1cb0e60700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac0ce6c0 con 0x7f1cac0b2510 2023-11-28T11:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1cb0e60700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac0cd460 con 0x7f1cac0cd090 2023-11-28T11:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1cb0e60700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac159d50 con 0x7f1cac0cf570 2023-11-28T11:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1caad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1cac0b2510 0x7f1cac0cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50694/0 (socket says 172.21.15.83:50694) 2023-11-28T11:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1caad9d700 1 -- 172.21.15.83:0/386302712 learned_addr learned my addr 172.21.15.83:0/386302712 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca9d9b700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 0x7f1cac15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907816319 0 0) 0x7f1cac0ce6c0 con 0x7f1cac0b2510 2023-11-28T11:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c940035b0 con 0x7f1cac0b2510 2023-11-28T11:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 518998150 0 0) 0x7f1c940035b0 con 0x7f1cac0b2510 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 msgr2=0x7f1cac15ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 0x7f1cac15ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 >> v1:172.21.15.90:6789/0 conn(0x7f1cac0cd090 legacy=0x7f1cac0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.901+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0cdab0 con 0x7f1cac0b2510 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f1ca40019f0 con 0x7f1cac0b2510 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 >> v1:172.21.15.83:6789/0 conn(0x7f1cac0b2510 legacy=0x7f1cac0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 0x7f1c94003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94003fb0 0x7f1c94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c94004d90 0x7f1c94005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400bac0 con 0x7f1c94004d90 2023-11-28T11:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400bc30 con 0x7f1c94003fb0 2023-11-28T11:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400e000 con 0x7f1cac0cf570 2023-11-28T11:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c94004d90 0x7f1c94005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caa59c700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 0x7f1c94003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca9d9b700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94003fb0 0x7f1c94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 -- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 msgr2=0x7f1c94003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0cf570 0x7f1c94003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 -- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94003fb0 msgr2=0x7f1c94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94003fb0 0x7f1c94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 -- 172.21.15.83:0/386302712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940035b0 con 0x7f1c94004d90 2023-11-28T11:11:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1caad9d700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c94004d90 0x7f1c94005200 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400a800 tx=0x7f1c94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b390fb271893b73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400a2f0 con 0x7f1c94004d90 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca4012620 con 0x7f1c94004d90 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.902+0000 7f1ca959a700 1 -- 172.21.15.83:0/386302712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40129d0 con 0x7f1c94004d90 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1cb0e60700 1 -- 172.21.15.83:0/386302712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c94004d90 msgr2=0x7f1c94005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/386302712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c94004d90 0x7f1c94005200 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400a800 tx=0x7f1c94005880 comp rx=0 tx=0).stop 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1caad9d700 1 -- 172.21.15.83:0/386302712 reap_dead start 2023-11-28T11:11:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1cb0e60700 1 -- 172.21.15.83:0/386302712 shutdown_connections 2023-11-28T11:11:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1cb0e60700 1 -- 172.21.15.83:0/386302712 >> 172.21.15.83:0/386302712 conn(0x7f1cac06e9f0 msgr2=0x7f1cac0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1cb0e60700 1 -- 172.21.15.83:0/386302712 shutdown_connections 2023-11-28T11:11:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.903+0000 7f1cb0e60700 1 -- 172.21.15.83:0/386302712 wait complete. 2023-11-28T11:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.904+0000 7f1cb0e60700 1 Processor -- start 2023-11-28T11:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.904+0000 7f1cb0e60700 1 -- start start 2023-11-28T11:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1cb0e60700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94004d90 0x7f1cac146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1cb0e60700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 0x7f1cac0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1cb0e60700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0b5490 0x7f1cac0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1cb0e60700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15a550 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1cb0e60700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac147340 con 0x7f1c94004d90 2023-11-28T11:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1cb0e60700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b3c60 con 0x7f1cac0b5490 2023-11-28T11:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 0x7f1cac0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 0x7f1cac0b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52844/0 (socket says 172.21.15.83:52844) 2023-11-28T11:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 -- 172.21.15.83:0/1357087688 learned_addr learned my addr 172.21.15.83:0/1357087688 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1caad9d700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0b5490 0x7f1cac0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1caa59c700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94004d90 0x7f1cac146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 -- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0b5490 msgr2=0x7f1cac0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0b5490 0x7f1cac0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 -- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94004d90 msgr2=0x7f1cac146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94004d90 0x7f1cac146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.905+0000 7f1ca9d9b700 1 -- 172.21.15.83:0/1357087688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0ce070 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.906+0000 7f1ca9d9b700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 0x7f1cac0b4f50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00119f0 tx=0x7f1ca0011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.906+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004300 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.906+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0004460 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.906+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac0cdd90 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.906+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000f950 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.906+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac0ce350 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.907+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac074320 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.907+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ca0025070 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.907+0000 7f1c9affd700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c8c0682f0 0x7f1c8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.908+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1ca0086330 con 0x7f1cac0c9bb0 2023-11-28T11:11:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.908+0000 7f1caa59c700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c8c0682f0 0x7f1c8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.908+0000 7f1caa59c700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c8c0682f0 0x7f1c8c06a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c007fd0 tx=0x7f1c9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:33.912+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ca0052a50 con 0x7f1cac0c9bb0 2023-11-28T11:11:34.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.137+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cac0b2280 con 0x7f1cac0c9bb0 2023-11-28T11:11:34.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.138+0000 7f1c9affd700 1 -- 172.21.15.83:0/1357087688 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f1ca0056070 con 0x7f1cac0c9bb0 2023-11-28T11:11:34.139 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:34.139 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:27.389584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:11:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c8c0682f0 msgr2=0x7f1c8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c8c0682f0 0x7f1c8c06a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c007fd0 tx=0x7f1c9c009040 comp rx=0 tx=0).stop 2023-11-28T11:11:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 msgr2=0x7f1cac0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 0x7f1cac0b4f50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00119f0 tx=0x7f1ca0011db0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 shutdown_connections 2023-11-28T11:11:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cac0b5490 0x7f1cac0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c94004d90 0x7f1cac146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c8c0682f0 0x7f1c8c06a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 --2- 172.21.15.83:0/1357087688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cac0c9bb0 0x7f1cac0b4f50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 >> 172.21.15.83:0/1357087688 conn(0x7f1cac06e9f0 msgr2=0x7f1cac0cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 shutdown_connections 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.140+0000 7f1cb0e60700 1 -- 172.21.15.83:0/1357087688 wait complete. 2023-11-28T11:11:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2023-11-28T11:11:34.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:11:24.379444+0000', 'modified': '2023-11-28T11:11:27.389584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6231}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6231': {'gid': 6231, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/3038492821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 3038492821}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3038492821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:11:34.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:11:34.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T11:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.681+0000 7f826addf700 1 Processor -- start 2023-11-28T11:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.681+0000 7f826addf700 1 -- start start 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.681+0000 7f826addf700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f82640d0dc0 0x7f826415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f826addf700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8264157140 con 0x7f8264156d70 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f826addf700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826415d9e0 con 0x7f826415d1a0 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f826addf700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826415dcc0 con 0x7f82640d0dc0 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8268b7b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8264156d70 0x7f82640d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50710/0 (socket says 172.21.15.83:50710) 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8268b7b700 1 -- 172.21.15.83:0/2891727129 learned_addr learned my addr 172.21.15.83:0/2891727129 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8263fff700 1 --2- 172.21.15.83:0/2891727129 >> v2:172.21.15.90:3300/0 conn(0x7f82640d0dc0 0x7f826415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f82637fe700 1 -- 172.21.15.83:0/2891727129 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713004549 0 0) 0x7f8264157140 con 0x7f8264156d70 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f82637fe700 1 -- 172.21.15.83:0/2891727129 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82540035b0 con 0x7f8264156d70 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f82637fe700 1 -- 172.21.15.83:0/2891727129 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201723512 0 0) 0x7f82540035b0 con 0x7f8264156d70 2023-11-28T11:11:34.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8263fff700 1 -- 172.21.15.83:0/2891727129 >> v1:172.21.15.90:6789/0 conn(0x7f826415d1a0 legacy=0x7f826406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8263fff700 1 -- 172.21.15.83:0/2891727129 >> v1:172.21.15.83:6789/0 conn(0x7f8264156d70 legacy=0x7f82640d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8263fff700 1 -- 172.21.15.83:0/2891727129 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826406bdb0 con 0x7f82640d0dc0 2023-11-28T11:11:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f8263fff700 1 --2- 172.21.15.83:0/2891727129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82640d0dc0 0x7f826415cc60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8250009840 tx=0x7f8250009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76253b75b432538b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.682+0000 7f82637fe700 1 -- 172.21.15.83:0/2891727129 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258001670 con 0x7f82640d0dc0 2023-11-28T11:11:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f82637fe700 1 -- 172.21.15.83:0/2891727129 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250004030 con 0x7f82640d0dc0 2023-11-28T11:11:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f82637fe700 1 -- 172.21.15.83:0/2891727129 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500043c0 con 0x7f82640d0dc0 2023-11-28T11:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 -- 172.21.15.83:0/2891727129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82640d0dc0 msgr2=0x7f826415cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 --2- 172.21.15.83:0/2891727129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82640d0dc0 0x7f826415cc60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8250009840 tx=0x7f8250009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 -- 172.21.15.83:0/2891727129 shutdown_connections 2023-11-28T11:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 --2- 172.21.15.83:0/2891727129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82640d0dc0 0x7f826415cc60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 -- 172.21.15.83:0/2891727129 >> 172.21.15.83:0/2891727129 conn(0x7f826406ea50 msgr2=0x7f8264070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 -- 172.21.15.83:0/2891727129 shutdown_connections 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.683+0000 7f826addf700 1 -- 172.21.15.83:0/2891727129 wait complete. 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.684+0000 7f826addf700 1 Processor -- start 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.684+0000 7f826addf700 1 -- start start 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826addf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8264156d70 0x7f82640ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826addf700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f826415d1a0 0x7f82640cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826addf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 0x7f82640b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826addf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640caed0 con 0x7f82640b5c40 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826addf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b3cb0 con 0x7f8264156d70 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826addf700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b3f50 con 0x7f826415d1a0 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826937c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 0x7f82640b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826937c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 0x7f82640b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52860/0 (socket says 172.21.15.83:52860) 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826937c700 1 -- 172.21.15.83:0/1158526889 learned_addr learned my addr 172.21.15.83:0/1158526889 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f8268b7b700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8264156d70 0x7f82640ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f8263fff700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f826415d1a0 0x7f82640cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826937c700 1 -- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f826415d1a0 msgr2=0x7f82640cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.685+0000 7f826937c700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f826415d1a0 0x7f82640cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f826937c700 1 -- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8264156d70 msgr2=0x7f82640ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f826937c700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8264156d70 0x7f82640ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f826937c700 1 -- 172.21.15.83:0/1158526889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500094a0 con 0x7f82640b5c40 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f826937c700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 0x7f82640b4d10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f825c00cb50 tx=0x7f825c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f82617fa700 1 -- 172.21.15.83:0/1158526889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c003c00 con 0x7f82640b5c40 2023-11-28T11:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f82617fa700 1 -- 172.21.15.83:0/1158526889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c003d60 con 0x7f82640b5c40 2023-11-28T11:11:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826406c370 con 0x7f82640b5c40 2023-11-28T11:11:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f82617fa700 1 -- 172.21.15.83:0/1158526889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00aa20 con 0x7f82640b5c40 2023-11-28T11:11:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.686+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826406c090 con 0x7f82640b5c40 2023-11-28T11:11:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.687+0000 7f82617fa700 1 -- 172.21.15.83:0/1158526889 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f825c010050 con 0x7f82640b5c40 2023-11-28T11:11:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.688+0000 7f82617fa700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82480682f0 0x7f824806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.688+0000 7f82617fa700 1 -- 172.21.15.83:0/1158526889 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f825c085d00 con 0x7f82640b5c40 2023-11-28T11:11:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.688+0000 7f8268b7b700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82480682f0 0x7f824806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.688+0000 7f8268b7b700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82480682f0 0x7f824806a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8258001eb0 tx=0x7f8258001e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.690+0000 7f8276f68b80 1 Processor -- start 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.690+0000 7f8276f68b80 1 -- start start 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.690+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f771c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.690+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.690+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.690+0000 7f8276f68b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f770afbc0 con 0x560f771ed130 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7711e350 con 0x560f771f0120 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f77117880 con 0x560f771ef810 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771efbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52864/0 (socket says 172.21.15.83:52864) 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 -- 172.21.15.83:0/3371158073 learned_addr learned my addr 172.21.15.83:0/3371158073 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8268b7b700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f771c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f826937c700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 -- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 msgr2=0x560f771c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f771c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 -- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 msgr2=0x560f7720b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 -- 172.21.15.83:0/3371158073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f76f995e0 con 0x560f771ed130 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8268b7b700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f771c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f826937c700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f8263fff700 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771efbe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8250009ea0 tx=0x7f8250010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db94f78833bdd8af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f824e7fc700 1 -- 172.21.15.83:0/3371158073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250020070 con 0x560f771ed130 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f824e7fc700 1 -- 172.21.15.83:0/3371158073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500042a0 con 0x560f771ed130 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.691+0000 7f824e7fc700 1 -- 172.21.15.83:0/3371158073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500038a0 con 0x560f771ed130 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 -- 172.21.15.83:0/3371158073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 msgr2=0x560f771efbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771efbe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8250009ea0 tx=0x7f8250010000 comp rx=0 tx=0).stop 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 -- 172.21.15.83:0/3371158073 shutdown_connections 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f771c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3371158073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771efbe0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 -- 172.21.15.83:0/3371158073 >> 172.21.15.83:0/3371158073 conn(0x560f771c2d60 msgr2=0x560f771c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 -- 172.21.15.83:0/3371158073 shutdown_connections 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.692+0000 7f8276f68b80 1 -- 172.21.15.83:0/3371158073 wait complete. 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 -- start start 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 Processor -- start 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f77203840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f7720db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f76f9b8d0 con 0x560f771ed130 2023-11-28T11:11:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.693+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f772046f0 con 0x560f771f0120 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f7720f790 con 0x560f771ef810 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8276f68b80 10 client.0 _ll_register_callbacks cb 0x560f76fbd240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f77203840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f77203840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52870/0 (socket says 172.21.15.83:52870) 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 -- 172.21.15.83:0/3549224352 learned_addr learned my addr 172.21.15.83:0/3549224352 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8263fff700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f7720db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f826937c700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 -- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 msgr2=0x560f7720db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f7720db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 -- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 msgr2=0x560f7720e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 -- 172.21.15.83:0/3549224352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826406bdb0 con 0x560f771ed130 2023-11-28T11:11:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8263fff700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f7720db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f826937c700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8268b7b700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f77203840 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8258013720 tx=0x7f8258000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b35b8e8db2ba9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f82437fe700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f82437fe700 1 -- 172.21.15.83:0/3549224352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800fec0 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f82437fe700 1 -- 172.21.15.83:0/3549224352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f825801b070 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f770e5140 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f82437fe700 1 -- 172.21.15.83:0/3549224352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f825802c3b0 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8276f68b80 10 client.6411 fetch_fsmap learned FSMap version 33 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x560f76f45d90 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.694+0000 7f82437fe700 1 -- 172.21.15.83:0/3549224352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f770e5140 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f82437fe700 1 -- 172.21.15.83:0/3549224352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f825802c510 con 0x560f771ed130 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8276f68b80 10 client.6411 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:11:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8276f68b80 10 client.6411 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8276f68b80 20 client.6411 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771e7890 0x560f77203190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8276f68b80 4 client.6411 mds_command: new command op to 6231 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f76f9bf10 con 0x560f771e7890 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8263fff700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771e7890 0x560f77203190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f82437fe700 10 client.6411 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.695+0000 7f8263fff700 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771e7890 0x560f77203190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3aa3a166af0c16d server_cookie=1dbd2830644266d in_seq=0 out_seq=0 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f82437fe700 1 -- 172.21.15.83:0/3549224352 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560f76f9bf10 con 0x560f771e7890 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f82437fe700 10 client.6411 handle_command_reply: tid=0 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 client.6411 shutdown 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 msgr2=0x560f77203840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f77203840 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8258013720 tx=0x7f8258000f40 comp rx=0 tx=0).stop 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 shutdown_connections 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771e7890 0x560f77203190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771f0120 0x560f7720e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f77203840 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 --2- 172.21.15.83:0/3549224352 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ef810 0x560f7720db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f82437fe700 0 client.6411 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 >> 172.21.15.83:0/3549224352 conn(0x560f771c2d60 msgr2=0x560f771c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 shutdown_connections 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 1 -- 172.21.15.83:0/3549224352 wait complete. 2023-11-28T11:11:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.696+0000 7f8276f68b80 20 client.6411 trim_cache size 0 max 16384 2023-11-28T11:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.713+0000 7f8276f68b80 1 Processor -- start 2023-11-28T11:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.713+0000 7f8276f68b80 1 -- start start 2023-11-28T11:11:34.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ed130 0x560f771c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f7720ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 0x560f771be450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8276f68b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f771c28a0 con 0x560f771f0120 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f771c71f0 con 0x560f771ef810 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f771bfd90 con 0x560f771ed130 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 0x560f771be450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 0x560f771be450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52882/0 (socket says 172.21.15.83:52882) 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 -- 172.21.15.83:0/4274728230 learned_addr learned my addr 172.21.15.83:0/4274728230 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8268b7b700 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ed130 0x560f771c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8263fff700 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f7720ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 -- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ed130 msgr2=0x560f771c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ed130 0x560f771c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 -- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 msgr2=0x560f7720ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f7720ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 -- 172.21.15.83:0/4274728230 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f770de140 con 0x560f771f0120 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f8263fff700 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f7720ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f826937c700 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 0x560f771be450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f825c055f90 tx=0x7f825c055ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc35a93edd21fe0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.714+0000 7f824effd700 1 -- 172.21.15.83:0/4274728230 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c055550 con 0x560f771f0120 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f824effd700 1 -- 172.21.15.83:0/4274728230 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c0556b0 con 0x560f771f0120 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f824effd700 1 -- 172.21.15.83:0/4274728230 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c055820 con 0x560f771f0120 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 -- 172.21.15.83:0/4274728230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 msgr2=0x560f771be450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 0x560f771be450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f825c055f90 tx=0x7f825c055ac0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 -- 172.21.15.83:0/4274728230 shutdown_connections 2023-11-28T11:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771f0120 0x560f771be450 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771ed130 0x560f771c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 --2- 172.21.15.83:0/4274728230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f7720ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 -- 172.21.15.83:0/4274728230 >> 172.21.15.83:0/4274728230 conn(0x560f771c2d60 msgr2=0x560f77211420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 -- 172.21.15.83:0/4274728230 shutdown_connections 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.715+0000 7f8276f68b80 1 -- 172.21.15.83:0/4274728230 wait complete. 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.716+0000 7f8276f68b80 1 -- start start 2023-11-28T11:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.716+0000 7f8276f68b80 1 Processor -- start 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f77258e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771f0120 0x560f771f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f771ee220 con 0x560f771ed130 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f771e63b0 con 0x560f771ef810 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f771f30d0 con 0x560f771f0120 2023-11-28T11:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 10 client.0 _ll_register_callbacks cb 0x560f76fbd240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771e5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52884/0 (socket says 172.21.15.83:52884) 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 -- 172.21.15.83:0/983741301 learned_addr learned my addr 172.21.15.83:0/983741301 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8263fff700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f77258e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f826937c700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771f0120 0x560f771f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 -- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771f0120 msgr2=0x560f771f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771f0120 0x560f771f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 -- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 msgr2=0x560f77258e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f77258e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 -- 172.21.15.83:0/983741301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826406c090 con 0x560f771ed130 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8268b7b700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771e5db0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8258013720 tx=0x7f8258014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3af3711d9ed98a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f82437fe700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f82437fe700 1 -- 172.21.15.83:0/983741301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800fea0 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f82437fe700 1 -- 172.21.15.83:0/983741301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f825802ca70 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f770d5160 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f82437fe700 1 -- 172.21.15.83:0/983741301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8258013b90 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 10 client.6417 fetch_fsmap learned FSMap version 33 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.717+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x560f76f6c200 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f82437fe700 1 -- 172.21.15.83:0/983741301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f770d5160 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f82437fe700 1 -- 172.21.15.83:0/983741301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8258012070 con 0x560f771ed130 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 10 client.6417 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 10 client.6417 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 20 client.6417 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771be450 0x560f771be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 4 client.6417 mds_command: new command op to 6231 tid=0[{"prefix": "status"}] 2023-11-28T11:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560f770bf170 con 0x560f771be450 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8263fff700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771be450 0x560f771be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f82437fe700 10 client.6417 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8263fff700 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771be450 0x560f771be820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae988b6e87aecff server_cookie=5943f65994b66488 in_seq=0 out_seq=0 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f82437fe700 1 -- 172.21.15.83:0/983741301 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560f770bf170 con 0x560f771be450 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f82437fe700 10 client.6417 handle_command_reply: tid=0 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.718+0000 7f8276f68b80 1 client.6417 shutdown 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 msgr2=0x560f771e5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771e5db0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8258013720 tx=0x7f8258014000 comp rx=0 tx=0).stop 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 shutdown_connections 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f771be450 0x560f771be820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f771f0120 0x560f771f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f771ed130 0x560f771e5db0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f82437fe700 0 client.6417 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 --2- 172.21.15.83:0/983741301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f771ef810 0x560f77258e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 >> 172.21.15.83:0/983741301 conn(0x560f771c2d60 msgr2=0x560f7720ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 shutdown_connections 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 1 -- 172.21.15.83:0/983741301 wait complete. 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f8276f68b80 20 client.6417 trim_cache size 0 max 16384 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "69424f07-94c3-4dbc-8507-3551775e0a09", 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6231, 2023-11-28T11:11:34.728 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 10.338748225, 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 33, 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 63, 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 63, 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 15.678914499999999 2023-11-28T11:11:34.729 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82480682f0 msgr2=0x7f824806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82480682f0 0x7f824806a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8258001eb0 tx=0x7f8258001e40 comp rx=0 tx=0).stop 2023-11-28T11:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 msgr2=0x7f82640b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 0x7f82640b4d10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f825c00cb50 tx=0x7f825c00ce60 comp rx=0 tx=0).stop 2023-11-28T11:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 shutdown_connections 2023-11-28T11:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82480682f0 0x7f824806a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82640b5c40 0x7f82640b4d10 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8264156d70 0x7f82640ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 --2- 172.21.15.83:0/1158526889 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f826415d1a0 0x7f82640cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 >> 172.21.15.83:0/1158526889 conn(0x7f82640d0dc0 msgr2=0x7f826406d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 shutdown_connections 2023-11-28T11:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.719+0000 7f826addf700 1 -- 172.21.15.83:0/1158526889 wait complete. 2023-11-28T11:11:34.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:11:34.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.851+0000 7f1bace67700 1 Processor -- start 2023-11-28T11:11:34.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.852+0000 7f1bace67700 1 -- start start 2023-11-28T11:11:34.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.852+0000 7f1bace67700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 0x7f1ba80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.852+0000 7f1bace67700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 0x7f1ba80b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.852+0000 7f1bace67700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba815efa0 con 0x7f1ba815a480 2023-11-28T11:11:34.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.852+0000 7f1bace67700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba80ceab0 con 0x7f1ba815a850 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.852+0000 7f1bace67700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80cecc0 con 0x7f1ba80b4270 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 0x7f1ba80b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba5d9b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 0x7f1ba80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba659c700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f1ba815a480 0x7f1ba80cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:49098/0 (socket says 172.21.15.83:49098) 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 0x7f1ba80b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:37098/0 (socket says 172.21.15.83:37098) 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba5d9b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 0x7f1ba80b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:39888/0 (socket says 172.21.15.83:39888) 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba659c700 1 -- 172.21.15.83:0/3996190077 learned_addr learned my addr 172.21.15.83:0/3996190077 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba559a700 1 -- 172.21.15.83:0/3996190077 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154251394 0 0) 0x7f1ba815efa0 con 0x7f1ba815a480 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/3996190077 >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 msgr2=0x7f1ba80b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 --2- 172.21.15.83:0/3996190077 >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 0x7f1ba80b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/3996190077 >> v1:172.21.15.90:6789/0 conn(0x7f1ba815a480 legacy=0x7f1ba80cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/3996190077 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba80cefa0 con 0x7f1ba80b4270 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba5d9b700 1 --2- 172.21.15.83:0/3996190077 >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 0x7f1ba80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:34.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba6d9d700 1 --2- 172.21.15.83:0/3996190077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 0x7f1ba80b2960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80cebf0 tx=0x7f1ba000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2068f78214ffb61f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba559a700 1 -- 172.21.15.83:0/3996190077 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0016070 con 0x7f1ba80b4270 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba559a700 1 -- 172.21.15.83:0/3996190077 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba000bb20 con 0x7f1ba80b4270 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.853+0000 7f1ba559a700 1 -- 172.21.15.83:0/3996190077 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000e070 con 0x7f1ba80b4270 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 -- 172.21.15.83:0/3996190077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 msgr2=0x7f1ba80b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 --2- 172.21.15.83:0/3996190077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 0x7f1ba80b2960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80cebf0 tx=0x7f1ba000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 -- 172.21.15.83:0/3996190077 shutdown_connections 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 --2- 172.21.15.83:0/3996190077 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba80b4270 0x7f1ba80b2960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 --2- 172.21.15.83:0/3996190077 >> v2:172.21.15.90:3300/0 conn(0x7f1ba815a850 0x7f1ba80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 -- 172.21.15.83:0/3996190077 >> 172.21.15.83:0/3996190077 conn(0x7f1ba806de20 msgr2=0x7f1ba806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 -- 172.21.15.83:0/3996190077 shutdown_connections 2023-11-28T11:11:34.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.854+0000 7f1bace67700 1 -- 172.21.15.83:0/3996190077 wait complete. 2023-11-28T11:11:34.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.855+0000 7f1bace67700 1 Processor -- start 2023-11-28T11:11:34.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.855+0000 7f1bace67700 1 -- start start 2023-11-28T11:11:34.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1bace67700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ba80b4270 0x7f1ba8157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1bace67700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba815a480 0x7f1ba806d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1bace67700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 0x7f1ba806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1bace67700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba806c610 con 0x7f1ba815a850 2023-11-28T11:11:34.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1bace67700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba8157ab0 con 0x7f1ba80b4270 2023-11-28T11:11:34.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1bace67700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba806bcb0 con 0x7f1ba815a480 2023-11-28T11:11:34.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1ba6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 0x7f1ba806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1ba6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 0x7f1ba806d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52902/0 (socket says 172.21.15.83:52902) 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/4236852098 learned_addr learned my addr 172.21.15.83:0/4236852098 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1ba659c700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ba80b4270 0x7f1ba8157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.856+0000 7f1ba5d9b700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba815a480 0x7f1ba806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba815a480 msgr2=0x7f1ba806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1ba6d9d700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba815a480 0x7f1ba806d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ba80b4270 msgr2=0x7f1ba8157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1ba6d9d700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ba80b4270 0x7f1ba8157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1ba6d9d700 1 -- 172.21.15.83:0/4236852098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980094a0 con 0x7f1ba815a850 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1ba6d9d700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 0x7f1ba806d590 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b370 tx=0x7f1ba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0016040 con 0x7f1ba815a850 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba0004800 con 0x7f1ba815a850 2023-11-28T11:11:34.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba0009a70 con 0x7f1ba815a850 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003ea0 con 0x7f1ba815a850 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.857+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba80cfeb0 con 0x7f1ba815a850 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.858+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba80cf280 con 0x7f1ba815a850 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.858+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ba002d070 con 0x7f1ba815a850 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.859+0000 7f1b96ffd700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b880682f0 0x7f1b8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.859+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1ba0087710 con 0x7f1ba815a850 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.859+0000 7f1ba659c700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b880682f0 0x7f1b8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.859+0000 7f1ba659c700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b880682f0 0x7f1b8806a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c007fd0 tx=0x7f1b9c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:34.863+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ba0053dd0 con 0x7f1ba815a850 2023-11-28T11:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.090+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ba80d0040 con 0x7f1ba815a850 2023-11-28T11:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.091+0000 7f1b96ffd700 1 -- 172.21.15.83:0/4236852098 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ba00573f0 con 0x7f1ba815a850 2023-11-28T11:11:35.092 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:11:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b880682f0 msgr2=0x7f1b8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b880682f0 0x7f1b8806a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c007fd0 tx=0x7f1b9c007de0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 msgr2=0x7f1ba806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 0x7f1ba806d590 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b370 tx=0x7f1ba0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 shutdown_connections 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b880682f0 0x7f1b8806a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba815a850 0x7f1ba806d590 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ba80b4270 0x7f1ba8157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 --2- 172.21.15.83:0/4236852098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1ba815a480 0x7f1ba806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 >> 172.21.15.83:0/4236852098 conn(0x7f1ba806de20 msgr2=0x7f1ba80cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 shutdown_connections 2023-11-28T11:11:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:35.093+0000 7f1bace67700 1 -- 172.21.15.83:0/4236852098 wait complete. 2023-11-28T11:11:35.107 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:11:35.108 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:35.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.108 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-28T11:11:35.126 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:11:35.127 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:11:35.127 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:11:35.127 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.127 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-28T11:11:35.127 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.128 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:35.128 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:35.128 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:35.128 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:35.128 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:35.128 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:35.129 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:11:35.129 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:35.129 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:11:35.129 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42178sec preferred_lft 42178sec 2023-11-28T11:11:35.129 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::22f1:9e4c:955b:b4a/64 scope link noprefixroute 2023-11-28T11:11:35.129 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.130 INFO:teuthology.orchestra.run.smithi083.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:11:35.130 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:35.130 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:11:35.130 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.130 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::80fe:9fff:fe99:6cb2/64 scope link 2023-11-28T11:11:35.130 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 1e:9b:f5:74:5a:be brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::1c9b:f5ff:fe74:5abe/64 scope link 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:d1:52:12:18:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:11:35.131 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5cd1:52ff:fe12:1849/64 scope link 2023-11-28T11:11:35.132 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:35.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.132 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:11:35.192 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-28T11:11:35.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.193 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-28T11:11:35.252 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-28T11:11:35.252 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:11:35.252 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:11:35.252 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:11:35.252 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:11:35.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.253 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-28T11:11:35.253 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-28T11:11:35.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:11:35.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:11:35.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:11:35.253 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-28T11:11:35.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.357 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:11:35.374 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:11:35.374 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:11:35.374 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:35.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-28T11:11:35.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:11:35.441 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:35.498 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:35.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:11:35.499 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:35.555 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-28T11:11:35.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:11:35.640 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:11:35.641 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:11:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:11:35.744 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.90:3300,172.21.15.90:3301" 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@69424f07-94c3-4dbc-8507-3551775e0a09.cephfs=/ 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.90:3300/172.21.15.90:3301" will pass to kernel 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790" 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2023-11-28T11:11:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=69424f07-94c3-4dbc-8507-3551775e0a09" will pass to kernel 2023-11-28T11:11:35.747 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:11:35.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:35.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.774 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:11:35.854 INFO:teuthology.orchestra.run.smithi083.stdout:client5884 2023-11-28T11:11:35.855 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:11:35.855 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:35.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.913 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T11:11:35.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:35.973 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T11:11:36.031 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:11:36.032 DEBUG:teuthology.orchestra.run.smithi083:> import os 2023-11-28T11:11:36.032 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2023-11-28T11:11:36.032 DEBUG:teuthology.orchestra.run.smithi083:> 2023-11-28T11:11:36.032 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T11:11:36.032 DEBUG:teuthology.orchestra.run.smithi083:> ' 2023-11-28T11:11:36.137 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2023-11-28T11:11:36.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.484+0000 7f6c946b4700 1 Processor -- start 2023-11-28T11:11:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.484+0000 7f6c946b4700 1 -- start start 2023-11-28T11:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.485+0000 7f6c946b4700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6c8c158c80 0x7f6c8c159de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.485+0000 7f6c946b4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c8c06dc60 0x7f6c8c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.485+0000 7f6c946b4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c15cdc0 con 0x7f6c8c0cd000 2023-11-28T11:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.485+0000 7f6c946b4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c15d890 con 0x7f6c8c06dc60 2023-11-28T11:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.485+0000 7f6c946b4700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c15cff0 con 0x7f6c8c158c80 2023-11-28T11:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92c51700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c8c06dc60 0x7f6c8c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c91c4f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6c8c0cd000 0x7f6c8c06d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50770/0 (socket says 172.21.15.83:50770) 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c91c4f700 1 -- 172.21.15.83:0/1801328523 learned_addr learned my addr 172.21.15.83:0/1801328523 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92450700 1 --2- 172.21.15.83:0/1801328523 >> v2:172.21.15.90:3300/0 conn(0x7f6c8c158c80 0x7f6c8c159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92c51700 1 -- 172.21.15.83:0/1801328523 >> v2:172.21.15.90:3300/0 conn(0x7f6c8c158c80 msgr2=0x7f6c8c159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92c51700 1 --2- 172.21.15.83:0/1801328523 >> v2:172.21.15.90:3300/0 conn(0x7f6c8c158c80 0x7f6c8c159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92c51700 1 -- 172.21.15.83:0/1801328523 >> v1:172.21.15.83:6789/0 conn(0x7f6c8c0cd000 legacy=0x7f6c8c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92c51700 1 -- 172.21.15.83:0/1801328523 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c15d2d0 con 0x7f6c8c06dc60 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c9144e700 1 -- 172.21.15.83:0/1801328523 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831385400 0 0) 0x7f6c8c15cdc0 con 0x7f6c8c0cd000 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c92c51700 1 --2- 172.21.15.83:0/1801328523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c15c880 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c070f10 tx=0x7f6c8400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca3727e8b21a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c9144e700 1 -- 172.21.15.83:0/1801328523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400f070 con 0x7f6c8c06dc60 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c9144e700 1 -- 172.21.15.83:0/1801328523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400a2e0 con 0x7f6c8c06dc60 2023-11-28T11:11:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.486+0000 7f6c9144e700 1 -- 172.21.15.83:0/1801328523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400cc00 con 0x7f6c8c06dc60 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 -- 172.21.15.83:0/1801328523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c06dc60 msgr2=0x7f6c8c15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1801328523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c15c880 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c070f10 tx=0x7f6c8400c330 comp rx=0 tx=0).stop 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 -- 172.21.15.83:0/1801328523 shutdown_connections 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1801328523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c15c880 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1801328523 >> v2:172.21.15.90:3300/0 conn(0x7f6c8c158c80 0x7f6c8c159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 -- 172.21.15.83:0/1801328523 >> 172.21.15.83:0/1801328523 conn(0x7f6c8c0cf430 msgr2=0x7f6c8c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 -- 172.21.15.83:0/1801328523 shutdown_connections 2023-11-28T11:11:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.487+0000 7f6c946b4700 1 -- 172.21.15.83:0/1801328523 wait complete. 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.488+0000 7f6c946b4700 1 Processor -- start 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.488+0000 7f6c946b4700 1 -- start start 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c8c0cd000 0x7f6c8c0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 0x7f6c8c0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c149db0 con 0x7f6c8c158c80 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c15ae10 con 0x7f6c8c06dc60 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c15b0f0 con 0x7f6c8c0cd000 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 0x7f6c8c0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 0x7f6c8c0c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52920/0 (socket says 172.21.15.83:52920) 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 -- 172.21.15.83:0/1575613818 learned_addr learned my addr 172.21.15.83:0/1575613818 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92450700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c91c4f700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c8c0cd000 0x7f6c8c0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 -- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c8c0cd000 msgr2=0x7f6c8c0c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c8c0cd000 0x7f6c8c0c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 -- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c8c06dc60 msgr2=0x7f6c8c149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 -- 172.21.15.83:0/1575613818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8400ab80 con 0x7f6c8c158c80 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c92c51700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 0x7f6c8c0c70c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c730 tx=0x7f6c84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004510 con 0x7f6c8c158c80 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c84004670 con 0x7f6c8c158c80 2023-11-28T11:11:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c06b080 con 0x7f6c8c158c80 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84008440 con 0x7f6c8c158c80 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.489+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c15d5b0 con 0x7f6c8c158c80 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.490+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c159050 con 0x7f6c8c158c80 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.491+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c84024030 con 0x7f6c8c158c80 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.491+0000 7f6c82ffd700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c700682f0 0x7f6c7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.491+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6c84087400 con 0x7f6c8c158c80 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.491+0000 7f6c92450700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c700682f0 0x7f6c7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.492+0000 7f6c92450700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c700682f0 0x7f6c7006a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00b840 tx=0x7f6c7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.495+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c84053aa0 con 0x7f6c8c158c80 2023-11-28T11:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.721+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c8c074320 con 0x7f6c8c158c80 2023-11-28T11:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.721+0000 7f6c82ffd700 1 -- 172.21.15.83:0/1575613818 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f6c840570c0 con 0x7f6c8c158c80 2023-11-28T11:11:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:36.726 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:27.389584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c700682f0 msgr2=0x7f6c7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c700682f0 0x7f6c7006a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00b840 tx=0x7f6c7c008040 comp rx=0 tx=0).stop 2023-11-28T11:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 msgr2=0x7f6c8c0c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 0x7f6c8c0c70c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c730 tx=0x7f6c84004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 shutdown_connections 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6c700682f0 0x7f6c7006a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c8c158c80 0x7f6c8c0c70c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c8c06dc60 0x7f6c8c149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 --2- 172.21.15.83:0/1575613818 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c8c0cd000 0x7f6c8c0c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 >> 172.21.15.83:0/1575613818 conn(0x7f6c8c0cf430 msgr2=0x7f6c8c155a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 shutdown_connections 2023-11-28T11:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.723+0000 7f6c946b4700 1 -- 172.21.15.83:0/1575613818 wait complete. 2023-11-28T11:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2023-11-28T11:11:36.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T11:11:36.883 INFO:teuthology.orchestra.run.smithi083.stdout:{"message":"","return_code":0} 2023-11-28T11:11:36.908 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:11:36.909 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:11:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.951+0000 7f3c6827f080 1 Processor -- start 2023-11-28T11:11:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.951+0000 7f3c6827f080 1 -- start start 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c6827f080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad5cce70 0x55e0ad6a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c6827f080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad6a4170 0x55e0ad69e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c6827f080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 0x55e0ad69e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c6827f080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ad506fe0 con 0x55e0ad615200 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c6827f080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ad5cc900 con 0x55e0ad6a4170 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c6827f080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0ad5ccd20 con 0x55e0ad5cce70 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 0x55e0ad69e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 0x55e0ad69e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52932/0 (socket says 172.21.15.83:52932) 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 -- 172.21.15.83:0/2852302285 learned_addr learned my addr 172.21.15.83:0/2852302285 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 -- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad5cce70 msgr2=0x55e0ad6a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5e382700 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad5cce70 0x55e0ad6a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5db81700 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad6a4170 0x55e0ad69e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad5cce70 0x55e0ad6a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 -- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad6a4170 msgr2=0x55e0ad69e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad6a4170 0x55e0ad69e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.952+0000 7f3c5eb83700 1 -- 172.21.15.83:0/2852302285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50005d20 con 0x55e0ad615200 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c5eb83700 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 0x55e0ad69e540 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3c58001660 tx=0x7f3c58001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9924bfd992ce2a00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c5cb7f700 1 -- 172.21.15.83:0/2852302285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58002120 con 0x55e0ad615200 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c5cb7f700 1 -- 172.21.15.83:0/2852302285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58003670 con 0x55e0ad615200 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c5cb7f700 1 -- 172.21.15.83:0/2852302285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58003a20 con 0x55e0ad615200 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 -- 172.21.15.83:0/2852302285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 msgr2=0x55e0ad69e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 0x55e0ad69e540 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3c58001660 tx=0x7f3c58001970 comp rx=0 tx=0).stop 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 -- 172.21.15.83:0/2852302285 shutdown_connections 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad615200 0x55e0ad69e540 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad5cce70 0x55e0ad6a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 --2- 172.21.15.83:0/2852302285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad6a4170 0x55e0ad69e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 -- 172.21.15.83:0/2852302285 >> 172.21.15.83:0/2852302285 conn(0x55e0ad617af0 msgr2=0x55e0ad5ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 -- 172.21.15.83:0/2852302285 shutdown_connections 2023-11-28T11:11:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.953+0000 7f3c6827f080 1 -- 172.21.15.83:0/2852302285 wait complete. 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.954+0000 7f3c6827f080 1 Processor -- start 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.954+0000 7f3c6827f080 1 -- start start 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad5cce70 0x55e0ad5ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad615200 0x55e0ad694490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad6a4170 0x55e0ad5fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ad610b50 con 0x55e0ad5cce70 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ad610220 con 0x55e0ad615200 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0ad610500 con 0x55e0ad6a4170 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad5cce70 0x55e0ad5ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad5cce70 0x55e0ad5ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52934/0 (socket says 172.21.15.83:52934) 2023-11-28T11:11:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 -- 172.21.15.83:0/3743914294 learned_addr learned my addr 172.21.15.83:0/3743914294 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5db81700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad615200 0x55e0ad694490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5eb83700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad6a4170 0x55e0ad5fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 -- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad6a4170 msgr2=0x55e0ad5fb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55e0ad6a4170 0x55e0ad5fb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 -- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad615200 msgr2=0x55e0ad694490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0ad615200 0x55e0ad694490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 -- 172.21.15.83:0/3743914294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58006c60 con 0x55e0ad5cce70 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c5e382700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad5cce70 0x55e0ad5ca570 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bd90 tx=0x7f3c540076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c4f7fe700 1 -- 172.21.15.83:0/3743914294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54010070 con 0x55e0ad5cce70 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c4f7fe700 1 -- 172.21.15.83:0/3743914294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5400b0d0 con 0x55e0ad5cce70 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e0ad51f120 con 0x55e0ad5cce70 2023-11-28T11:11:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c4f7fe700 1 -- 172.21.15.83:0/3743914294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014610 con 0x55e0ad5cce70 2023-11-28T11:11:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.955+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0ad512830 con 0x55e0ad5cce70 2023-11-28T11:11:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c4f7fe700 1 -- 172.21.15.83:0/3743914294 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3c5401b030 con 0x55e0ad5cce70 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c4f7fe700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3c40066c10 0x7f3c400690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c4f7fe700 1 -- 172.21.15.83:0/3743914294 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3c54083c00 con 0x55e0ad5cce70 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c5db81700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3c40066c10 0x7f3c400690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c6827f080 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55e0ad73a340 0x55e0ad6a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63) v8 -- 0x55e0ad6a2140 con 0x55e0ad73a340 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c5eb83700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55e0ad73a340 0x55e0ad6a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.957+0000 7f3c5db81700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3c40066c10 0x7f3c400690c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3c50015040 tx=0x7f3c5000f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c5eb83700 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55e0ad73a340 0x55e0ad6a1c00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c5eb83700 1 -- 172.21.15.83:0/3743914294 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f3c580037e0 con 0x55e0ad73a340 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55e0ad73a340 msgr2=0x55e0ad6a1c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x55e0ad73a340 0x55e0ad6a1c00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3c40066c10 msgr2=0x7f3c400690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3c40066c10 0x7f3c400690c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3c50015040 tx=0x7f3c5000f000 comp rx=0 tx=0).stop 2023-11-28T11:11:36.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad5cce70 msgr2=0x55e0ad5ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 --2- 172.21.15.83:0/3743914294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ad5cce70 0x55e0ad5ca570 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bd90 tx=0x7f3c540076d0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c5eb83700 1 -- 172.21.15.83:0/3743914294 reap_dead start 2023-11-28T11:11:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 shutdown_connections 2023-11-28T11:11:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 >> 172.21.15.83:0/3743914294 conn(0x55e0ad617af0 msgr2=0x55e0ad695570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 shutdown_connections 2023-11-28T11:11:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:36.958+0000 7f3c6827f080 1 -- 172.21.15.83:0/3743914294 wait complete. 2023-11-28T11:11:36.967 INFO:teuthology.orchestra.run.smithi083.stdout:Xfiledir_test 2023-11-28T11:11:36.967 DEBUG:teuthology.orchestra.run.smithi083:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: "ancestors": [ 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1099511627776, 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "file", 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 2 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:11:37.063 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1, 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "dir_test", 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 4 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 11, 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: "old_pools": [] 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:37.064 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:11:37.065 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T11:11:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.148+0000 7f5bdb1b3080 1 Processor -- start 2023-11-28T11:11:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.148+0000 7f5bdb1b3080 1 -- start start 2023-11-28T11:11:37.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bdb1b3080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bdd4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bdb1b3080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd38630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bdb1b3080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bcbd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bdb1b3080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7bbaffe0 con 0x560c7bd4c790 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bdb1b3080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7bca0e60 con 0x560c7bd4e390 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bdb1b3080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c7bca1130 con 0x560c7bd4e760 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bdd4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bdd4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52950/0 (socket says 172.21.15.83:52950) 2023-11-28T11:11:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2257578558 learned_addr learned my addr 172.21.15.83:0/2257578558 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd1ab7700 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bcbd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 msgr2=0x560c7bcbd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd0ab5700 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd38630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bcbd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 msgr2=0x560c7bd38630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd38630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2257578558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc00a100 con 0x560c7bd4c790 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.149+0000 7f5bd12b6700 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bdd4de0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8001c70 tx=0x7f5bc800b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b67b427bd2c5264 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bc37fe700 1 -- 172.21.15.83:0/2257578558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800be60 con 0x560c7bd4c790 2023-11-28T11:11:37.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bc37fe700 1 -- 172.21.15.83:0/2257578558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc800e470 con 0x560c7bd4c790 2023-11-28T11:11:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bc37fe700 1 -- 172.21.15.83:0/2257578558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800e820 con 0x560c7bd4c790 2023-11-28T11:11:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2257578558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 msgr2=0x560c7bdd4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bdd4de0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8001c70 tx=0x7f5bc800b5d0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2257578558 shutdown_connections 2023-11-28T11:11:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bcbd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bdd4de0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2257578558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd38630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2257578558 >> 172.21.15.83:0/2257578558 conn(0x560c7bcc0b00 msgr2=0x560c7bc74ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2257578558 shutdown_connections 2023-11-28T11:11:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.150+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2257578558 wait complete. 2023-11-28T11:11:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 Processor -- start 2023-11-28T11:11:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 -- start start 2023-11-28T11:11:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bd3f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd42ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bd3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7bd44170 con 0x560c7bd4c790 2023-11-28T11:11:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7bd434f0 con 0x560c7bd4e390 2023-11-28T11:11:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bdb1b3080 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c7bd437d0 con 0x560c7bd4e760 2023-11-28T11:11:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.151+0000 7f5bd12b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bd3f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bd3f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52956/0 (socket says 172.21.15.83:52956) 2023-11-28T11:11:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2500866206 learned_addr learned my addr 172.21.15.83:0/2500866206 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd0ab5700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd42ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd1ab7700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bd3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 msgr2=0x560c7bd3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bd3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 msgr2=0x560c7bd42ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd42ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 -- 172.21.15.83:0/2500866206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c7bbbb4a0 con 0x560c7bd4c790 2023-11-28T11:11:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd0ab5700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c7bd4e390 0x560c7bd42ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd1ab7700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560c7bd4e760 0x560c7bd3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bd12b6700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bd3f420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x560c7bdd5de0 tx=0x7f5bc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bc27fc700 1 -- 172.21.15.83:0/2500866206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc801e070 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bc27fc700 1 -- 172.21.15.83:0/2500866206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc800ee70 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c7bbc8120 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bc27fc700 1 -- 172.21.15.83:0/2500866206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8023620 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.152+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c7bbbb830 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.153+0000 7f5bc27fc700 1 -- 172.21.15.83:0/2500866206 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5bc8029020 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bc27fc700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5bb4066c10 0x7f5bb40690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bc27fc700 1 -- 172.21.15.83:0/2500866206 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5bc8086370 con 0x560c7bd4c790 2023-11-28T11:11:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bd0ab5700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5bb4066c10 0x7f5bb40690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x560c7bd44aa0 0x560c7bd36ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63) v8 -- 0x560c7bd37010 con 0x560c7bd44aa0 2023-11-28T11:11:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bd1ab7700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x560c7bd44aa0 0x560c7bd36ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bd0ab5700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5bb4066c10 0x7f5bb40690c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x560c7bd45830 tx=0x7f5bc4005d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.154+0000 7f5bd1ab7700 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x560c7bd44aa0 0x560c7bd36ad0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bd1ab7700 1 -- 172.21.15.83:0/2500866206 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v63'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bcc00fa60 con 0x560c7bd44aa0 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x560c7bd44aa0 msgr2=0x560c7bd36ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x560c7bd44aa0 0x560c7bd36ad0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5bb4066c10 msgr2=0x7f5bb40690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f5bb4066c10 0x7f5bb40690c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x560c7bd45830 tx=0x7f5bc4005d10 comp rx=0 tx=0).stop 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 msgr2=0x560c7bd3f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 --2- 172.21.15.83:0/2500866206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c7bd4c790 0x560c7bd3f420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x560c7bdd5de0 tx=0x7f5bc8000bc0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bd1ab7700 1 -- 172.21.15.83:0/2500866206 reap_dead start 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 shutdown_connections 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 >> 172.21.15.83:0/2500866206 conn(0x560c7bcc0b00 msgr2=0x560c7bcbd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 shutdown_connections 2023-11-28T11:11:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.156+0000 7f5bdb1b3080 1 -- 172.21.15.83:0/2500866206 wait complete. 2023-11-28T11:11:37.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T11:11:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 Processor -- start 2023-11-28T11:11:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 -- start start 2023-11-28T11:11:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76e40719a0 0x7f76e406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e4158fa0 con 0x7f76e4158bd0 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e406db40 con 0x7f76e40719a0 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.291+0000 7f76e9764700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4071e20 con 0x7f76e40c4a70 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76e40719a0 0x7f76e406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76e40719a0 0x7f76e406df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52970/0 (socket says 172.21.15.83:52970) 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e2ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f76e4158bd0 0x7f76e40cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50826/0 (socket says 172.21.15.83:50826) 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 -- 172.21.15.83:0/1360773371 learned_addr learned my addr 172.21.15.83:0/1360773371 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e27fc700 1 --2- 172.21.15.83:0/1360773371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 -- 172.21.15.83:0/1360773371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 msgr2=0x7f76e4071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 --2- 172.21.15.83:0/1360773371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 -- 172.21.15.83:0/1360773371 >> v1:172.21.15.83:6789/0 conn(0x7f76e4158bd0 legacy=0x7f76e40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 -- 172.21.15.83:0/1360773371 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e40ce2c0 con 0x7f76e40719a0 2023-11-28T11:11:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e1ffb700 1 -- 172.21.15.83:0/1360773371 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626750972 0 0) 0x7f76e4158fa0 con 0x7f76e4158bd0 2023-11-28T11:11:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e37fe700 1 --2- 172.21.15.83:0/1360773371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e40719a0 0x7f76e406df60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009e50 tx=0x7f76dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c67dbf55bccfef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e1ffb700 1 -- 172.21.15.83:0/1360773371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00e070 con 0x7f76e40719a0 2023-11-28T11:11:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e1ffb700 1 -- 172.21.15.83:0/1360773371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc00b9c0 con 0x7f76e40719a0 2023-11-28T11:11:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.292+0000 7f76e1ffb700 1 -- 172.21.15.83:0/1360773371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00bd70 con 0x7f76e40719a0 2023-11-28T11:11:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 -- 172.21.15.83:0/1360773371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e40719a0 msgr2=0x7f76e406df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 --2- 172.21.15.83:0/1360773371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e40719a0 0x7f76e406df60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009e50 tx=0x7f76dc00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 -- 172.21.15.83:0/1360773371 shutdown_connections 2023-11-28T11:11:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 --2- 172.21.15.83:0/1360773371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e40719a0 0x7f76e406df60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 --2- 172.21.15.83:0/1360773371 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 -- 172.21.15.83:0/1360773371 >> 172.21.15.83:0/1360773371 conn(0x7f76e40b1390 msgr2=0x7f76e40ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 -- 172.21.15.83:0/1360773371 shutdown_connections 2023-11-28T11:11:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.293+0000 7f76e9764700 1 -- 172.21.15.83:0/1360773371 wait complete. 2023-11-28T11:11:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.294+0000 7f76e9764700 1 Processor -- start 2023-11-28T11:11:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.294+0000 7f76e9764700 1 -- start start 2023-11-28T11:11:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e9764700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e40719a0 0x7f76e414c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e9764700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e9764700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 0x7f76e406a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e9764700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e414ce00 con 0x7f76e4158bd0 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e9764700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e41523d0 con 0x7f76e40719a0 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e9764700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e41526b0 con 0x7f76e40c4a70 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 0x7f76e406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 0x7f76e406a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52976/0 (socket says 172.21.15.83:52976) 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 -- 172.21.15.83:0/4118156039 learned_addr learned my addr 172.21.15.83:0/4118156039 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e40719a0 0x7f76e414c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e27fc700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 -- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 msgr2=0x7f76e4151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 -- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e40719a0 msgr2=0x7f76e414c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e40719a0 0x7f76e414c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 -- 172.21.15.83:0/4118156039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc009b30 con 0x7f76e4158bd0 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76e37fe700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 0x7f76e406a270 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f76e4159130 tx=0x7f76dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.295+0000 7f76cb7fe700 1 -- 172.21.15.83:0/4118156039 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc003ea0 con 0x7f76e4158bd0 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.296+0000 7f76cb7fe700 1 -- 172.21.15.83:0/4118156039 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc0048e0 con 0x7f76e4158bd0 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.296+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e406d760 con 0x7f76e4158bd0 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.296+0000 7f76cb7fe700 1 -- 172.21.15.83:0/4118156039 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc01b670 con 0x7f76e4158bd0 2023-11-28T11:11:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.296+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e40ce5a0 con 0x7f76e4158bd0 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.297+0000 7f76cb7fe700 1 -- 172.21.15.83:0/4118156039 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f76dc02b070 con 0x7f76e4158bd0 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.297+0000 7f76cb7fe700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76d00682f0 0x7f76d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.297+0000 7f76cb7fe700 1 -- 172.21.15.83:0/4118156039 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f76dc087490 con 0x7f76e4158bd0 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.297+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76d00682f0 0x7f76d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.298+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76d00682f0 0x7f76d006a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76d4007fd0 tx=0x7f76d4007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.299+0000 7f76f58edb80 1 Processor -- start 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 -- start start 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612993078c0 0x561299307c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x5612993081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612991c7c70 con 0x5612992dc270 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x561299236400 con 0x5612993078c0 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56129922f930 con 0x561299300d90 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76e27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x5612993081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76e27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x5612993081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52982/0 (socket says 172.21.15.83:52982) 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.300+0000 7f76e27fc700 1 -- 172.21.15.83:0/2554669448 learned_addr learned my addr 172.21.15.83:0/2554669448 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612993078c0 0x561299307c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e37fe700 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e27fc700 1 -- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561299300d90 msgr2=0x561299301200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e27fc700 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e27fc700 1 -- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612993078c0 msgr2=0x561299307c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e27fc700 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612993078c0 0x561299307c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e27fc700 1 -- 172.21.15.83:0/2554669448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612990b1690 con 0x5612992dc270 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76e27fc700 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x5612993081d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f76d80099a0 tx=0x7f76d8009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb73fe212ac4ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76c8ff9700 1 -- 172.21.15.83:0/2554669448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d800c070 con 0x5612992dc270 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76c8ff9700 1 -- 172.21.15.83:0/2554669448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8004230 con 0x5612992dc270 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.301+0000 7f76c8ff9700 1 -- 172.21.15.83:0/2554669448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d80116f0 con 0x5612992dc270 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 -- 172.21.15.83:0/2554669448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 msgr2=0x5612993081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x5612993081d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f76d80099a0 tx=0x7f76d8009cb0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 -- 172.21.15.83:0/2554669448 shutdown_connections 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612993078c0 0x561299307c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 --2- 172.21.15.83:0/2554669448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x5612993081d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 -- 172.21.15.83:0/2554669448 >> 172.21.15.83:0/2554669448 conn(0x5612992dae10 msgr2=0x5612992db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 -- 172.21.15.83:0/2554669448 shutdown_connections 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 -- 172.21.15.83:0/2554669448 wait complete. 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 -- start start 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.302+0000 7f76f58edb80 1 Processor -- start 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612992dc270 0x5612992fe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612993078c0 0x561299301740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612990b3980 con 0x561299300d90 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612993260e0 con 0x5612992dc270 2023-11-28T11:11:37.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612992fe8d0 con 0x5612993078c0 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76f58edb80 10 client.0 _ll_register_callbacks cb 0x5612990d52f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52988/0 (socket says 172.21.15.83:52988) 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 -- 172.21.15.83:0/3965622853 learned_addr learned my addr 172.21.15.83:0/3965622853 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612992dc270 0x5612992fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e37fe700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612993078c0 0x561299301740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 -- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612993078c0 msgr2=0x561299301740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612993078c0 0x561299301740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 -- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612992dc270 msgr2=0x5612992fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612992dc270 0x5612992fe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.303+0000 7f76e27fc700 1 -- 172.21.15.83:0/3965622853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d800e040 con 0x561299300d90 2023-11-28T11:11:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76e27fc700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 0x561299301200 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000c00 tx=0x7f76d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f614f1c6d9ca8714 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76c1ffb700 1 -- 172.21.15.83:0/3965622853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8011b00 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76c1ffb700 1 -- 172.21.15.83:0/3965622853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f76d8011c60 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612991fd1f0 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76c1ffb700 1 -- 172.21.15.83:0/3965622853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d8011e20 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 10 client.6468 fetch_fsmap learned FSMap version 33 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x56129905de40 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76c1ffb700 1 -- 172.21.15.83:0/3965622853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612991fd1f0 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76c1ffb700 1 -- 172.21.15.83:0/3965622853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f76d8026070 con 0x561299300d90 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 10 client.6468 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 10 client.6468 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T11:11:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 20 client.6468 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612992dd650 0x561299302b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 4 client.6468 mds_command: new command op to 6231 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612990b3fc0 con 0x5612992dd650 2023-11-28T11:11:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.304+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612992dd650 0x561299302b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76c1ffb700 10 client.6468 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612992dd650 0x561299302b00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a34c7c3b5b4e8e server_cookie=4b482a2bb071c6e4 in_seq=0 out_seq=0 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76c1ffb700 1 -- 172.21.15.83:0/3965622853 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5612990b3fc0 con 0x5612992dd650 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76c1ffb700 10 client.6468 handle_command_reply: tid=0 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 client.6468 shutdown 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 msgr2=0x561299301200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 0x561299301200 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000c00 tx=0x7f76d80047c0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 shutdown_connections 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612992dd650 0x561299302b00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612993078c0 0x561299301740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612992dc270 0x5612992fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76c1ffb700 0 client.6468 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 --2- 172.21.15.83:0/3965622853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561299300d90 0x561299301200 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 >> 172.21.15.83:0/3965622853 conn(0x5612992dae10 msgr2=0x5612992db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 shutdown_connections 2023-11-28T11:11:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.305+0000 7f76f58edb80 1 -- 172.21.15.83:0/3965622853 wait complete. 2023-11-28T11:11:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.306+0000 7f76f58edb80 20 client.6468 trim_cache size 0 max 16384 2023-11-28T11:11:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.322+0000 7f76f58edb80 1 Processor -- start 2023-11-28T11:11:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.322+0000 7f76f58edb80 1 -- start start 2023-11-28T11:11:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dae10 0x561299303360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x561299318450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 0x561299318990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76f58edb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612992d2b30 con 0x5612992dc270 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612992de050 con 0x561299300d90 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x561299339820 con 0x5612992dae10 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x561299318450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x561299318450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52996/0 (socket says 172.21.15.83:52996) 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 -- 172.21.15.83:0/501208392 learned_addr learned my addr 172.21.15.83:0/501208392 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 -- 172.21.15.83:0/501208392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dae10 msgr2=0x561299303360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dae10 0x561299303360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 -- 172.21.15.83:0/501208392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 msgr2=0x561299318990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 0x561299318990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 -- 172.21.15.83:0/501208392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612991f61f0 con 0x5612992dc270 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76e27fc700 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x561299318450 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f76d8004840 tx=0x7f76d801f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f1cf3fcaf3ffa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76c97fa700 1 -- 172.21.15.83:0/501208392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d802f440 con 0x5612992dc270 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76c97fa700 1 -- 172.21.15.83:0/501208392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8023040 con 0x5612992dc270 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.323+0000 7f76c97fa700 1 -- 172.21.15.83:0/501208392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8011b20 con 0x5612992dc270 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 -- 172.21.15.83:0/501208392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 msgr2=0x561299318450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x561299318450 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f76d8004840 tx=0x7f76d801f070 comp rx=0 tx=0).stop 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 -- 172.21.15.83:0/501208392 shutdown_connections 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 0x561299318990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dae10 0x561299303360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 --2- 172.21.15.83:0/501208392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dc270 0x561299318450 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 -- 172.21.15.83:0/501208392 >> 172.21.15.83:0/501208392 conn(0x561299323c50 msgr2=0x5612992fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 -- 172.21.15.83:0/501208392 shutdown_connections 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 -- 172.21.15.83:0/501208392 wait complete. 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 -- start start 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.324+0000 7f76f58edb80 1 Processor -- start 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 0x5612993906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dc270 0x5612992dc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 0x5612992dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612993297a0 con 0x5612992dae10 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129932af00 con 0x561299300d90 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612993276b0 con 0x5612992dc270 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76f58edb80 10 client.0 _ll_register_callbacks cb 0x5612990d52f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:37.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76e2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 0x5612993906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76e2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 0x5612993906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53000/0 (socket says 172.21.15.83:53000) 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.325+0000 7f76e2ffd700 1 -- 172.21.15.83:0/1650146176 learned_addr learned my addr 172.21.15.83:0/1650146176 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e2ffd700 1 -- 172.21.15.83:0/1650146176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dc270 msgr2=0x5612992dc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dc270 0x5612992dc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e2ffd700 1 -- 172.21.15.83:0/1650146176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 msgr2=0x5612992dcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 0x5612992dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e2ffd700 1 -- 172.21.15.83:0/1650146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56129905de40 con 0x5612992dae10 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e2ffd700 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 0x5612993906f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000f60 tx=0x7f76d4001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6431ec16826a02b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76c1ffb700 1 -- 172.21.15.83:0/1650146176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4004c50 con 0x5612992dae10 2023-11-28T11:11:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76c1ffb700 1 -- 172.21.15.83:0/1650146176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f76d401b040 con 0x5612992dae10 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612991ed210 con 0x5612992dae10 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76c1ffb700 1 -- 172.21.15.83:0/1650146176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d4023760 con 0x5612992dae10 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 10 client.6474 fetch_fsmap learned FSMap version 33 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5612990842b0 con 0x5612992dae10 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76c1ffb700 1 -- 172.21.15.83:0/1650146176 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612991ed210 con 0x5612992dae10 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76c1ffb700 1 -- 172.21.15.83:0/1650146176 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f76d40238c0 con 0x5612992dae10 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 10 client.6474 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 10 client.6474 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 20 client.6474 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612993078c0 0x56129931a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 4 client.6474 mds_command: new command op to 6231 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T11:11:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x5612991d7220 con 0x5612993078c0 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.326+0000 7f76e27fc700 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612993078c0 0x56129931a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76c1ffb700 10 client.6474 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76e27fc700 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612993078c0 0x56129931a840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f26ed1aec95bc69 server_cookie=bbeaf9038f5b9375 in_seq=0 out_seq=0 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76c1ffb700 1 -- 172.21.15.83:0/1650146176 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5612991d7220 con 0x5612993078c0 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76c1ffb700 10 client.6474 handle_command_reply: tid=0 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 client.6474 shutdown 2023-11-28T11:11:37.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 msgr2=0x5612993906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 0x5612993906f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000f60 tx=0x7f76d4001080 comp rx=0 tx=0).stop 2023-11-28T11:11:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 shutdown_connections 2023-11-28T11:11:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x5612993078c0 0x56129931a840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561299300d90 0x5612992dcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612992dae10 0x5612993906f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76c1ffb700 0 client.6474 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 --2- 172.21.15.83:0/1650146176 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x5612992dc270 0x5612992dc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 >> 172.21.15.83:0/1650146176 conn(0x561299323c50 msgr2=0x5612992db1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.327+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 shutdown_connections 2023-11-28T11:11:37.338 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f20a9861700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5884=0-4194304@1} caps={5884=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5884 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562f95c13b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76f58edb80 1 -- 172.21.15.83:0/1650146176 wait complete. 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76f58edb80 20 client.6474 trim_cache size 0 max 16384 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "fbedb878-d3f9-454c-8499-5a5aeb7d74c6", 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2023-11-28T11:11:37.339 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76d00682f0 msgr2=0x7f76d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76d00682f0 0x7f76d006a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76d4007fd0 tx=0x7f76d4007de0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 msgr2=0x7f76e406a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 0x7f76e406a270 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f76e4159130 tx=0x7f76dc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 shutdown_connections 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f76d00682f0 0x7f76d006a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e4158bd0 0x7f76e406a270 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e40719a0 0x7f76e414c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 --2- 172.21.15.83:0/4118156039 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76e40c4a70 0x7f76e4151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 >> 172.21.15.83:0/4118156039 conn(0x7f76e40b1390 msgr2=0x7f76e40cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 shutdown_connections 2023-11-28T11:11:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.328+0000 7f76e9764700 1 -- 172.21.15.83:0/4118156039 wait complete. 2023-11-28T11:11:37.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:11:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.463+0000 7fed8d395700 1 Processor -- start 2023-11-28T11:11:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.463+0000 7fed8d395700 1 -- start start 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed8d395700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fed880cb540 0x7fed8815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed8d395700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 0x7fed880ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed8d395700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8815e620 con 0x7fed8815a520 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed8d395700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8815f0f0 con 0x7fed880cb540 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed8d395700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c9fa0 con 0x7fed880ca3c0 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed86ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fed8815a520 0x7fed880b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50868/0 (socket says 172.21.15.83:50868) 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed86ffd700 1 -- 172.21.15.83:0/3017965152 learned_addr learned my addr 172.21.15.83:0/3017965152 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:11:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.464+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739315149 0 0) 0x7fed8815e620 con 0x7fed8815a520 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 0x7fed880ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed700035b0 con 0x7fed8815a520 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed867fc700 1 --2- 172.21.15.83:0/3017965152 >> v2:172.21.15.90:3300/0 conn(0x7fed880cb540 0x7fed8815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044458059 0 0) 0x7fed700035b0 con 0x7fed8815a520 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 msgr2=0x7fed880ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 0x7fed880ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 >> v2:172.21.15.90:3300/0 conn(0x7fed880cb540 msgr2=0x7fed8815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 --2- 172.21.15.83:0/3017965152 >> v2:172.21.15.90:3300/0 conn(0x7fed880cb540 0x7fed8815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8815e850 con 0x7fed8815a520 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fed7c001670 con 0x7fed8815a520 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 >> v1:172.21.15.83:6789/0 conn(0x7fed8815a520 legacy=0x7fed880b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 0x7fed70003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70003f80 0x7fed70004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70004d90 0x7fed70005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815f0f0 con 0x7fed70004d90 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c9fa0 con 0x7fed70003f80 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7000ba40 con 0x7fed880ca3c0 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 0x7fed70003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed867fc700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70003f80 0x7fed70004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70004d90 0x7fed70005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 -- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 msgr2=0x7fed70003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed880ca3c0 0x7fed70003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 -- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70003f80 msgr2=0x7fed70004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70003f80 0x7fed70004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 -- 172.21.15.83:0/3017965152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed700035b0 con 0x7fed70004d90 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.465+0000 7fed877fe700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70004d90 0x7fed70005200 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fed80009fc0 tx=0x7fed70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe6db9a1b1c4f02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000ebd0 con 0x7fed70004d90 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000f040 con 0x7fed70004d90 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed85ffb700 1 -- 172.21.15.83:0/3017965152 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80014480 con 0x7fed70004d90 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed8d395700 1 -- 172.21.15.83:0/3017965152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70004d90 msgr2=0x7fed70005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed8d395700 1 --2- 172.21.15.83:0/3017965152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70004d90 0x7fed70005200 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fed80009fc0 tx=0x7fed70005880 comp rx=0 tx=0).stop 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed877fe700 1 -- 172.21.15.83:0/3017965152 reap_dead start 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed8d395700 1 -- 172.21.15.83:0/3017965152 shutdown_connections 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed8d395700 1 -- 172.21.15.83:0/3017965152 >> 172.21.15.83:0/3017965152 conn(0x7fed880cf300 msgr2=0x7fed880cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed8d395700 1 -- 172.21.15.83:0/3017965152 shutdown_connections 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.466+0000 7fed8d395700 1 -- 172.21.15.83:0/3017965152 wait complete. 2023-11-28T11:11:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 Processor -- start 2023-11-28T11:11:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 -- start start 2023-11-28T11:11:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed880aeab0 0x7fed881e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed881e4990 0x7fed88157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 0x7fed88157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c7e90 con 0x7fed881e4480 2023-11-28T11:11:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c8170 con 0x7fed880aeab0 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed8d395700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c8450 con 0x7fed881e4990 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed877fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 0x7fed88157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed877fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 0x7fed88157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53022/0 (socket says 172.21.15.83:53022) 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.468+0000 7fed877fe700 1 -- 172.21.15.83:0/4210479436 learned_addr learned my addr 172.21.15.83:0/4210479436 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed867fc700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed881e4990 0x7fed88157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed86ffd700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed880aeab0 0x7fed881e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed877fe700 1 -- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed881e4990 msgr2=0x7fed88157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed877fe700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed881e4990 0x7fed88157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed877fe700 1 -- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed880aeab0 msgr2=0x7fed881e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed877fe700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed880aeab0 0x7fed881e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed877fe700 1 -- 172.21.15.83:0/4210479436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8815ee10 con 0x7fed881e4480 2023-11-28T11:11:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed867fc700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed881e4990 0x7fed88157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed877fe700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 0x7fed88157a60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fed80016000 tx=0x7fed80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed6f7fe700 1 -- 172.21.15.83:0/4210479436 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80007000 con 0x7fed881e4480 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed6f7fe700 1 -- 172.21.15.83:0/4210479436 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80022070 con 0x7fed881e4480 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8815eb30 con 0x7fed881e4480 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed6f7fe700 1 -- 172.21.15.83:0/4210479436 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80029480 con 0x7fed881e4480 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.469+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8815a8f0 con 0x7fed881e4480 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.471+0000 7fed6f7fe700 1 -- 172.21.15.83:0/4210479436 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed8001e430 con 0x7fed881e4480 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.471+0000 7fed6f7fe700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fed740682f0 0x7fed7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.471+0000 7fed6f7fe700 1 -- 172.21.15.83:0/4210479436 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed80024070 con 0x7fed881e4480 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.471+0000 7fed86ffd700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fed740682f0 0x7fed7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.472+0000 7fed86ffd700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fed740682f0 0x7fed7406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0048d0 tx=0x7fed7c007980 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.473+0000 7fed9951eb80 1 Processor -- start 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.473+0000 7fed9951eb80 1 -- start start 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 0x556f38eeac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ebf210 0x556f38eeb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38ee41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed9951eb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38daac10 con 0x556f38eea860 2023-11-28T11:11:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38e193a0 con 0x556f38ebf210 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f38e128d0 con 0x556f38ee3d30 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 0x556f38eeac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 0x556f38eeac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53026/0 (socket says 172.21.15.83:53026) 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 -- 172.21.15.83:0/3772104011 learned_addr learned my addr 172.21.15.83:0/3772104011 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed867fc700 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ebf210 0x556f38eeb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed877fe700 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38ee41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 -- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 msgr2=0x556f38ee41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38ee41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 -- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ebf210 msgr2=0x556f38eeb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ebf210 0x556f38eeb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 -- 172.21.15.83:0/3772104011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f38c94630 con 0x556f38eea860 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.474+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 0x556f38eeac30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007440 tx=0x7fed7c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c152ab052f6f691 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed6cff9700 1 -- 172.21.15.83:0/3772104011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c01a040 con 0x556f38eea860 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed6cff9700 1 -- 172.21.15.83:0/3772104011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c007800 con 0x556f38eea860 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed6cff9700 1 -- 172.21.15.83:0/3772104011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c015ce0 con 0x556f38eea860 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 -- 172.21.15.83:0/3772104011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 msgr2=0x556f38eeac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 0x556f38eeac30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007440 tx=0x7fed7c00c000 comp rx=0 tx=0).stop 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 -- 172.21.15.83:0/3772104011 shutdown_connections 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38ee41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38eea860 0x556f38eeac30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3772104011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ebf210 0x556f38eeb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 -- 172.21.15.83:0/3772104011 >> 172.21.15.83:0/3772104011 conn(0x556f38ebddb0 msgr2=0x556f38ebe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 -- 172.21.15.83:0/3772104011 shutdown_connections 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.475+0000 7fed9951eb80 1 -- 172.21.15.83:0/3772104011 wait complete. 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.476+0000 7fed9951eb80 1 -- start start 2023-11-28T11:11:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.476+0000 7fed9951eb80 1 Processor -- start 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f11dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38edca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38eea860 0x556f38f07240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38c96920 con 0x556f38ebf210 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38f0ff20 con 0x556f38eea860 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f38f11680 con 0x556f38ee3d30 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed9951eb80 10 client.0 _ll_register_callbacks cb 0x556f38cb8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f11dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f11dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53032/0 (socket says 172.21.15.83:53032) 2023-11-28T11:11:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 -- 172.21.15.83:0/3210803751 learned_addr learned my addr 172.21.15.83:0/3210803751 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed867fc700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38edca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed877fe700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38eea860 0x556f38f07240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 -- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 msgr2=0x556f38edca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38edca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 -- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38eea860 msgr2=0x556f38f07240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38eea860 0x556f38f07240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 -- 172.21.15.83:0/3210803751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8815e850 con 0x556f38ebf210 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f11dd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x556f38ec23d0 tx=0x7fed7c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea096c15790c00ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.477+0000 7fed65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed65ffb700 1 -- 172.21.15.83:0/3210803751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c022dc0 con 0x556f38ebf210 2023-11-28T11:11:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed65ffb700 1 -- 172.21.15.83:0/3210803751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed7c015e30 con 0x556f38ebf210 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f38de0190 con 0x556f38ebf210 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed65ffb700 1 -- 172.21.15.83:0/3210803751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed7c007800 con 0x556f38ebf210 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 10 client.6486 fetch_fsmap learned FSMap version 33 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x556f38c40de0 con 0x556f38ebf210 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed65ffb700 1 -- 172.21.15.83:0/3210803751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f38de0190 con 0x556f38ebf210 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed65ffb700 1 -- 172.21.15.83:0/3210803751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fed7c020040 con 0x556f38ebf210 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 10 client.6486 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 10 client.6486 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 20 client.6486 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38edd1b0 0x556f38eeba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 4 client.6486 mds_command: new command op to 6231 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f38c96f60 con 0x556f38edd1b0 2023-11-28T11:11:37.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed867fc700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38edd1b0 0x556f38eeba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed65ffb700 10 client.6486 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.478+0000 7fed867fc700 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38edd1b0 0x556f38eeba20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48255097730822e5 server_cookie=8bd883d0380be51d in_seq=0 out_seq=0 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed65ffb700 1 -- 172.21.15.83:0/3210803751 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x556f38c96f60 con 0x556f38edd1b0 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed65ffb700 10 client.6486 handle_command_reply: tid=0 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 client.6486 shutdown 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 msgr2=0x556f38f11dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f11dd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x556f38ec23d0 tx=0x7fed7c000fe0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 shutdown_connections 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38edd1b0 0x556f38eeba20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38eea860 0x556f38f07240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed65ffb700 0 client.6486 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f11dd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3210803751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38ee3d30 0x556f38edca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 >> 172.21.15.83:0/3210803751 conn(0x556f38ebddb0 msgr2=0x556f38eeac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 shutdown_connections 2023-11-28T11:11:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 1 -- 172.21.15.83:0/3210803751 wait complete. 2023-11-28T11:11:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.479+0000 7fed9951eb80 20 client.6486 trim_cache size 0 max 16384 2023-11-28T11:11:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.495+0000 7fed9951eb80 1 Processor -- start 2023-11-28T11:11:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.495+0000 7fed9951eb80 1 -- start start 2023-11-28T11:11:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f0f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38eeae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38ee2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed9951eb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38eea4c0 con 0x556f38ebf210 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38ee3860 con 0x556f38ee3d30 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f38ebd390 con 0x556f38eea860 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f0f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53040/0 (socket says 172.21.15.83:53040) 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 -- 172.21.15.83:0/3200049111 learned_addr learned my addr 172.21.15.83:0/3200049111 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed867fc700 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38eeae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed877fe700 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38ee2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 -- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 msgr2=0x556f38ee2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38ee2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 -- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 msgr2=0x556f38eeae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.496+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38eeae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed86ffd700 1 -- 172.21.15.83:0/3200049111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f38dd9190 con 0x556f38ebf210 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed877fe700 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38ee2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f0f180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fed7c02d6d0 tx=0x7fed7c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c0b2ec1b76a03 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed6d7fa700 1 -- 172.21.15.83:0/3200049111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c022830 con 0x556f38ebf210 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed6d7fa700 1 -- 172.21.15.83:0/3200049111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c015720 con 0x556f38ebf210 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed6d7fa700 1 -- 172.21.15.83:0/3200049111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c022dc0 con 0x556f38ebf210 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 -- 172.21.15.83:0/3200049111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 msgr2=0x556f38f0f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f0f180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fed7c02d6d0 tx=0x7fed7c02c000 comp rx=0 tx=0).stop 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 -- 172.21.15.83:0/3200049111 shutdown_connections 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38ee2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f0f180 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3200049111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38eeae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 -- 172.21.15.83:0/3200049111 >> 172.21.15.83:0/3200049111 conn(0x556f38ebddb0 msgr2=0x556f38f07ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 -- 172.21.15.83:0/3200049111 shutdown_connections 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.497+0000 7fed9951eb80 1 -- 172.21.15.83:0/3200049111 wait complete. 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.498+0000 7fed9951eb80 1 -- start start 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.498+0000 7fed9951eb80 1 Processor -- start 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38ebe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38f6eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38f6f8c0 con 0x556f38ebf210 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f38ebe9b0 con 0x556f38ee3d30 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f38ebec00 con 0x556f38eea860 2023-11-28T11:11:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 10 client.0 _ll_register_callbacks cb 0x556f38cb8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f54800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53046/0 (socket says 172.21.15.83:53046) 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 -- 172.21.15.83:0/3415139916 learned_addr learned my addr 172.21.15.83:0/3415139916 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed877fe700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38f6eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed867fc700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38ebe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 -- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 msgr2=0x556f38f6eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38f6eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 -- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 msgr2=0x556f38ebe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38ebe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 -- 172.21.15.83:0/3415139916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f38c40de0 con 0x556f38ebf210 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed86ffd700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f54800 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fed7c030040 tx=0x7fed7c006030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71b3c20dda70d8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed65ffb700 1 -- 172.21.15.83:0/3415139916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c02d900 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed65ffb700 1 -- 172.21.15.83:0/3415139916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed7c02d440 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.499+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f38dd01b0 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed65ffb700 1 -- 172.21.15.83:0/3415139916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed7c02da60 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 10 client.6492 fetch_fsmap learned FSMap version 33 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x556f38c67250 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed65ffb700 1 -- 172.21.15.83:0/3415139916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f38dd01b0 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed65ffb700 1 -- 172.21.15.83:0/3415139916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fed7c020050 con 0x556f38ebf210 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 10 client.6492 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 10 client.6492 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 20 client.6492 populate_metadata read hostname 'smithi083' 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38f07370 0x556f38f6f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:37.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 4 client.6492 mds_command: new command op to 6231 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f38dba1c0 con 0x556f38f07370 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed867fc700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38f07370 0x556f38f6f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed65ffb700 10 client.6492 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed867fc700 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38f07370 0x556f38f6f000 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12eb5967e703a1f server_cookie=f8f567840aec49a9 in_seq=0 out_seq=0 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed65ffb700 1 -- 172.21.15.83:0/3415139916 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x556f38dba1c0 con 0x556f38f07370 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed65ffb700 10 client.6492 handle_command_reply: tid=0 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.500+0000 7fed9951eb80 1 client.6492 shutdown 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 msgr2=0x556f38f54800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f54800 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fed7c030040 tx=0x7fed7c006030 comp rx=0 tx=0).stop 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 shutdown_connections 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x556f38f07370 0x556f38f6f000 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x556f38eea860 0x556f38f6eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f38ebf210 0x556f38f54800 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 --2- 172.21.15.83:0/3415139916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556f38ee3d30 0x556f38ebe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed65ffb700 0 client.6492 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 >> 172.21.15.83:0/3415139916 conn(0x556f38ebddb0 msgr2=0x556f38eeae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 shutdown_connections 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 1 -- 172.21.15.83:0/3415139916 wait complete. 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed9951eb80 20 client.6492 trim_cache size 0 max 16384 2023-11-28T11:11:37.510 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: "fbedb878-d3f9-454c-8499-5a5aeb7d74c6": { 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "fbedb878-d3f9-454c-8499-5a5aeb7d74c6", 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:11:37.511 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:11:37.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fed740682f0 msgr2=0x7fed7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fed740682f0 0x7fed7406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0048d0 tx=0x7fed7c007980 comp rx=0 tx=0).stop 2023-11-28T11:11:37.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 msgr2=0x7fed88157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:37.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 0x7fed88157a60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fed80016000 tx=0x7fed80009750 comp rx=0 tx=0).stop 2023-11-28T11:11:37.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 shutdown_connections 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fed740682f0 0x7fed7406a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed881e4480 0x7fed88157a60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed880aeab0 0x7fed881e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 --2- 172.21.15.83:0/4210479436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fed881e4990 0x7fed88157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 >> 172.21.15.83:0/4210479436 conn(0x7fed880cf300 msgr2=0x7fed880b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 shutdown_connections 2023-11-28T11:11:37.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:11:37.501+0000 7fed8d395700 1 -- 172.21.15.83:0/4210479436 wait complete. 2023-11-28T11:11:37.514 INFO:tasks.cephfs.filesystem:scrub status for tag:fbedb878-d3f9-454c-8499-5a5aeb7d74c6 - {'path': '/', 'tag': 'fbedb878-d3f9-454c-8499-5a5aeb7d74c6', 'options': 'recursive'} 2023-11-28T11:11:39.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:39.998 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:40.850 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-28T11:11:40.849+0000 7fdf5c1f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T11:12:07.516 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:12:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.650+0000 7f460bc57700 1 Processor -- start 2023-11-28T11:12:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.651+0000 7f460bc57700 1 -- start start 2023-11-28T11:12:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.651+0000 7f460bc57700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46040b4ff0 0x7f46040cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f460bc57700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4604160bc0 0x7f46040d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f460bc57700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46040b26e0 con 0x7f46040b87e0 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f460bc57700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46040d0a30 con 0x7f46040b4ff0 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f460bc57700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46040d0d10 con 0x7f4604160bc0 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46040b4ff0 0x7f46040cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46040b4ff0 0x7f46040cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53056/0 (socket says 172.21.15.83:53056) 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 -- 172.21.15.83:0/21303960 learned_addr learned my addr 172.21.15.83:0/21303960 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f460a1f4700 1 --2- 172.21.15.83:0/21303960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4604160bc0 0x7f46040d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 -- 172.21.15.83:0/21303960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4604160bc0 msgr2=0x7f46040d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 --2- 172.21.15.83:0/21303960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4604160bc0 0x7f46040d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 -- 172.21.15.83:0/21303960 >> v1:172.21.15.90:6789/0 conn(0x7f46040b87e0 legacy=0x7f46040b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 -- 172.21.15.83:0/21303960 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604161040 con 0x7f46040b4ff0 2023-11-28T11:12:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46091f2700 1 --2- 172.21.15.83:0/21303960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46040b4ff0 0x7f46040cf0b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f46000098f0 tx=0x7f4600009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb873809a6ee407 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46089f1700 1 -- 172.21.15.83:0/21303960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460000b8a0 con 0x7f46040b4ff0 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46089f1700 1 -- 172.21.15.83:0/21303960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460000ba00 con 0x7f46040b4ff0 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.652+0000 7f46089f1700 1 -- 172.21.15.83:0/21303960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460000bd90 con 0x7f46040b4ff0 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 -- 172.21.15.83:0/21303960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46040b4ff0 msgr2=0x7f46040cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 --2- 172.21.15.83:0/21303960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46040b4ff0 0x7f46040cf0b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f46000098f0 tx=0x7f4600009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 -- 172.21.15.83:0/21303960 shutdown_connections 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 --2- 172.21.15.83:0/21303960 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4604160bc0 0x7f46040d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 --2- 172.21.15.83:0/21303960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46040b4ff0 0x7f46040cf0b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 -- 172.21.15.83:0/21303960 >> 172.21.15.83:0/21303960 conn(0x7f460406cef0 msgr2=0x7f46040cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 -- 172.21.15.83:0/21303960 shutdown_connections 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.653+0000 7f460bc57700 1 -- 172.21.15.83:0/21303960 wait complete. 2023-11-28T11:12:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.654+0000 7f460bc57700 1 Processor -- start 2023-11-28T11:12:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.654+0000 7f460bc57700 1 -- start start 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f460bc57700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46040b87e0 0x7f4604159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f460bc57700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 0x7f46040c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f460bc57700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f460415a440 0x7f46041595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f460bc57700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040c7eb0 con 0x7f4604160bc0 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f460bc57700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040c8190 con 0x7f46040b87e0 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f460bc57700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46040c8470 con 0x7f460415a440 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 0x7f46040c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 0x7f46040c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53062/0 (socket says 172.21.15.83:53062) 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 -- 172.21.15.83:0/4254928742 learned_addr learned my addr 172.21.15.83:0/4254928742 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 -- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f460415a440 msgr2=0x7f46041595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46099f3700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46040b87e0 0x7f4604159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f460415a440 0x7f46041595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 -- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46040b87e0 msgr2=0x7f4604159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46040b87e0 0x7f4604159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.655+0000 7f46091f2700 1 -- 172.21.15.83:0/4254928742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4600009550 con 0x7f4604160bc0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.656+0000 7f46091f2700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 0x7f46040c8f50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4600000c00 tx=0x7f4600004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.656+0000 7f45f67fc700 1 -- 172.21.15.83:0/4254928742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46000036a0 con 0x7f4604160bc0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.656+0000 7f45f67fc700 1 -- 172.21.15.83:0/4254928742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460001e070 con 0x7f4604160bc0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.656+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46040cd750 con 0x7f4604160bc0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.656+0000 7f45f67fc700 1 -- 172.21.15.83:0/4254928742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003a40 con 0x7f4604160bc0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.656+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46040cd4b0 con 0x7f4604160bc0 2023-11-28T11:12:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.657+0000 7f45f67fc700 1 -- 172.21.15.83:0/4254928742 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f460000f050 con 0x7f4604160bc0 2023-11-28T11:12:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.657+0000 7f45f67fc700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45e80682f0 0x7f45e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.658+0000 7f45f67fc700 1 -- 172.21.15.83:0/4254928742 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4600020070 con 0x7f4604160bc0 2023-11-28T11:12:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.658+0000 7f46099f3700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45e80682f0 0x7f45e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.658+0000 7f46099f3700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45e80682f0 0x7f45e806a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f45f8007fd0 tx=0x7f45f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.660+0000 7f4617de0b80 1 Processor -- start 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.660+0000 7f4617de0b80 1 -- start start 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c8f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c9137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f4617de0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c7b7c10 con 0x560f6c8f8170 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c8263a0 con 0x560f6c8f5180 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f6c81f8d0 con 0x560f6c8f7860 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c9137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c9137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53072/0 (socket says 172.21.15.83:53072) 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 -- 172.21.15.83:0/1257726620 learned_addr learned my addr 172.21.15.83:0/1257726620 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f46099f3700 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f46091f2700 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c8f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 -- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 msgr2=0x560f6c8cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 -- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 msgr2=0x560f6c8f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c8f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 -- 172.21.15.83:0/1257726620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6c6a1630 con 0x560f6c8f8170 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f460a1f4700 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c9137c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007330 tx=0x7f45fc007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7ac473e407e936 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f45e77fe700 1 -- 172.21.15.83:0/1257726620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc014600 con 0x560f6c8f8170 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f45e77fe700 1 -- 172.21.15.83:0/1257726620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc014760 con 0x560f6c8f8170 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.661+0000 7f45e77fe700 1 -- 172.21.15.83:0/1257726620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc014af0 con 0x560f6c8f8170 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 -- 172.21.15.83:0/1257726620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 msgr2=0x560f6c9137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c9137c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007330 tx=0x7f45fc007640 comp rx=0 tx=0).stop 2023-11-28T11:12:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 -- 172.21.15.83:0/1257726620 shutdown_connections 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c9137c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 --2- 172.21.15.83:0/1257726620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c8f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 -- 172.21.15.83:0/1257726620 >> 172.21.15.83:0/1257726620 conn(0x560f6c8cadb0 msgr2=0x560f6c8cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 -- 172.21.15.83:0/1257726620 shutdown_connections 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.662+0000 7f4617de0b80 1 -- 172.21.15.83:0/1257726620 wait complete. 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.663+0000 7f4617de0b80 1 -- start start 2023-11-28T11:12:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.663+0000 7f4617de0b80 1 Processor -- start 2023-11-28T11:12:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 0x560f6c8fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f7860 0x560f6c9183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c8effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c6a3920 con 0x560f6c8f5180 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c8f1b90 con 0x560f6c8f7860 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f6c918e50 con 0x560f6c8f8170 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f4617de0b80 10 client.0 _ll_register_callbacks cb 0x560f6c6c5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f46099f3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 0x560f6c8fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f46099f3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 0x560f6c8fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53074/0 (socket says 172.21.15.83:53074) 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.664+0000 7f46099f3700 1 -- 172.21.15.83:0/2443669366 learned_addr learned my addr 172.21.15.83:0/2443669366 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46091f2700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f7860 0x560f6c9183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f460a1f4700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c8effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46099f3700 1 -- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 msgr2=0x560f6c8effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46099f3700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c8effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46099f3700 1 -- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f7860 msgr2=0x560f6c9183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46099f3700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f7860 0x560f6c9183b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46099f3700 1 -- 172.21.15.83:0/2443669366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc009be0 con 0x560f6c8f5180 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46091f2700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f7860 0x560f6c9183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46099f3700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 0x560f6c8fb0a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f45f8014c90 tx=0x7f45f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd1599e1361ceac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f45e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2443669366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8012df0 con 0x560f6c8f5180 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2443669366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f45f8020d20 con 0x560f6c8f5180 2023-11-28T11:12:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f6c7ed190 con 0x560f6c8f5180 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2443669366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f8005110 con 0x560f6c8f5180 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 10 client.6504 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x560f6c64dde0 con 0x560f6c8f5180 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2443669366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f6c7ed190 con 0x560f6c8f5180 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2443669366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f45f8005270 con 0x560f6c8f5180 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 10 client.6504 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 10 client.6504 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 20 client.6504 populate_metadata read hostname 'smithi083' 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c918940 0x560f6c916d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 4 client.6504 mds_command: new command op to 6231 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f6c6a3f60 con 0x560f6c918940 2023-11-28T11:12:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.665+0000 7f46091f2700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c918940 0x560f6c916d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f45e4ff9700 10 client.6504 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f46091f2700 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c918940 0x560f6c916d40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27878d224a67ddac server_cookie=f596702af8e63b7a in_seq=0 out_seq=0 2023-11-28T11:12:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2443669366 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560f6c6a3f60 con 0x560f6c918940 2023-11-28T11:12:07.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f45e4ff9700 10 client.6504 handle_command_reply: tid=0 2023-11-28T11:12:07.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 client.6504 shutdown 2023-11-28T11:12:07.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 msgr2=0x560f6c8fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 0x560f6c8fb0a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f45f8014c90 tx=0x7f45f8000ed0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 shutdown_connections 2023-11-28T11:12:07.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c918940 0x560f6c916d40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c8effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f45e4ff9700 0 client.6504 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f5180 0x560f6c8fb0a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2443669366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f7860 0x560f6c9183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.666+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 >> 172.21.15.83:0/2443669366 conn(0x560f6c8cadb0 msgr2=0x560f6c8cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.667+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 shutdown_connections 2023-11-28T11:12:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.667+0000 7f4617de0b80 1 -- 172.21.15.83:0/2443669366 wait complete. 2023-11-28T11:12:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.667+0000 7f4617de0b80 20 client.6504 trim_cache size 0 max 16384 2023-11-28T11:12:07.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.683+0000 7f4617de0b80 1 Processor -- start 2023-11-28T11:12:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.683+0000 7f4617de0b80 1 -- start start 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c916090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c8ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f4617de0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c916fd0 con 0x560f6c8f8170 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c96ea90 con 0x560f6c8f5180 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f6c8c8760 con 0x560f6c8f7860 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f460a1f4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c8ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f460a1f4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c8ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53086/0 (socket says 172.21.15.83:53086) 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.684+0000 7f460a1f4700 1 -- 172.21.15.83:0/2872115040 learned_addr learned my addr 172.21.15.83:0/2872115040 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f46099f3700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c916090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f460a1f4700 1 -- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 msgr2=0x560f6c8cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f46091f2700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f460a1f4700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f460a1f4700 1 -- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 msgr2=0x560f6c916090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f460a1f4700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c916090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f460a1f4700 1 -- 172.21.15.83:0/2872115040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6c7e6190 con 0x560f6c8f8170 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f46099f3700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c916090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f46091f2700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f460a1f4700 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c8ccc50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f45fc017040 tx=0x7f45fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26428e141df1f853 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f45e7fff700 1 -- 172.21.15.83:0/2872115040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc003c20 con 0x560f6c8f8170 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f45e7fff700 1 -- 172.21.15.83:0/2872115040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc00d040 con 0x560f6c8f8170 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f45e7fff700 1 -- 172.21.15.83:0/2872115040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc004040 con 0x560f6c8f8170 2023-11-28T11:12:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 -- 172.21.15.83:0/2872115040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 msgr2=0x560f6c8ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c8ccc50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f45fc017040 tx=0x7f45fc012000 comp rx=0 tx=0).stop 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 -- 172.21.15.83:0/2872115040 shutdown_connections 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f8170 0x560f6c8ccc50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c916090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2872115040 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f7860 0x560f6c8cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 -- 172.21.15.83:0/2872115040 >> 172.21.15.83:0/2872115040 conn(0x560f6c8cadb0 msgr2=0x560f6c9192f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.685+0000 7f4617de0b80 1 -- 172.21.15.83:0/2872115040 shutdown_connections 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.686+0000 7f4617de0b80 1 -- 172.21.15.83:0/2872115040 wait complete. 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.686+0000 7f4617de0b80 1 -- start start 2023-11-28T11:12:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.686+0000 7f4617de0b80 1 Processor -- start 2023-11-28T11:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c96d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 0x560f6c90b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c90bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c8cd5c0 con 0x560f6c8f7860 2023-11-28T11:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6c9790c0 con 0x560f6c8f5180 2023-11-28T11:12:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f6c90c1b0 con 0x560f6c8f8170 2023-11-28T11:12:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f4617de0b80 10 client.0 _ll_register_callbacks cb 0x560f6c6c5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 0x560f6c90b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 0x560f6c90b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53090/0 (socket says 172.21.15.83:53090) 2023-11-28T11:12:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 -- 172.21.15.83:0/2282924424 learned_addr learned my addr 172.21.15.83:0/2282924424 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46099f3700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c96d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f460a1f4700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c90bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 -- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 msgr2=0x560f6c90bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c90bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 -- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 msgr2=0x560f6c96d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c96d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.687+0000 7f46091f2700 1 -- 172.21.15.83:0/2282924424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f6c6a1630 con 0x560f6c8f7860 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f46091f2700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 0x560f6c90b5f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f460000c010 tx=0x7f460000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8fd61d5227ca9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2282924424 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460005f280 con 0x560f6c8f7860 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2282924424 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f460005f3e0 con 0x560f6c8f7860 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f6c7dd1b0 con 0x560f6c8f7860 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2282924424 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4600003ba0 con 0x560f6c8f7860 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 10 client.6510 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x560f6c674250 con 0x560f6c8f7860 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2282924424 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f6c7dd1b0 con 0x560f6c8f7860 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2282924424 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4600062020 con 0x560f6c8f7860 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 10 client.6510 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 10 client.6510 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 20 client.6510 populate_metadata read hostname 'smithi083' 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c8cb4d0 0x560f6c9781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 4 client.6510 mds_command: new command op to 6231 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f6c7c71c0 con 0x560f6c8cb4d0 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f46099f3700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c8cb4d0 0x560f6c9781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f45e4ff9700 10 client.6510 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.688+0000 7f46099f3700 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c8cb4d0 0x560f6c9781a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7f57b292b300f server_cookie=5eb77366411636e2 in_seq=0 out_seq=0 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f45e4ff9700 1 -- 172.21.15.83:0/2282924424 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560f6c7c71c0 con 0x560f6c8cb4d0 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f45e4ff9700 10 client.6510 handle_command_reply: tid=0 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 client.6510 shutdown 2023-11-28T11:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 msgr2=0x560f6c90b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 0x560f6c90b5f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f460000c010 tx=0x7f460000bda0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 shutdown_connections 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x560f6c8cb4d0 0x560f6c9781a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x560f6c8f8170 0x560f6c90bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f45e4ff9700 0 client.6510 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f6c8f5180 0x560f6c96d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 --2- 172.21.15.83:0/2282924424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f6c8f7860 0x560f6c90b5f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 >> 172.21.15.83:0/2282924424 conn(0x560f6c8cadb0 msgr2=0x560f6c916090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 shutdown_connections 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 1 -- 172.21.15.83:0/2282924424 wait complete. 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f4617de0b80 20 client.6510 trim_cache size 0 max 16384 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45e80682f0 msgr2=0x7f45e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.689+0000 7f460bc57700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45e80682f0 0x7f45e806a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f45f8007fd0 tx=0x7f45f8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 msgr2=0x7f46040c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 0x7f46040c8f50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4600000c00 tx=0x7f4600004750 comp rx=0 tx=0).stop 2023-11-28T11:12:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 shutdown_connections 2023-11-28T11:12:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f460415a440 0x7f46041595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46040b87e0 0x7f4604159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f45e80682f0 0x7f45e806a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 --2- 172.21.15.83:0/4254928742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4604160bc0 0x7f46040c8f50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 >> 172.21.15.83:0/4254928742 conn(0x7f46040b4ff0 msgr2=0x7f46040b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 shutdown_connections 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.690+0000 7f460bc57700 1 -- 172.21.15.83:0/4254928742 wait complete. 2023-11-28T11:12:07.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:12:07.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.829+0000 7f7bbc516700 1 Processor -- start 2023-11-28T11:12:07.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.829+0000 7f7bbc516700 1 -- start start 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbc516700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 0x7f7bb40cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbc516700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7bb40cb660 0x7f7bb40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbc516700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb40cf330 con 0x7f7bb415a180 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbc516700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40cdfe0 con 0x7f7bb40cb660 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbc516700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4159d60 con 0x7f7bb40cb290 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbaab3700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7bb415a180 0x7f7bb415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50960/0 (socket says 172.21.15.83:50960) 2023-11-28T11:12:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bbaab3700 1 -- 172.21.15.83:0/3491762173 learned_addr learned my addr 172.21.15.83:0/3491762173 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 0x7f7bb40cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252430930 0 0) 0x7f7bb40cf330 con 0x7f7bb415a180 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba00035b0 con 0x7f7bb415a180 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb9ab1700 1 --2- 172.21.15.83:0/3491762173 >> v2:172.21.15.90:3300/0 conn(0x7f7bb40cb660 0x7f7bb40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3693408463 0 0) 0x7f7ba00035b0 con 0x7f7bb415a180 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 msgr2=0x7f7bb40cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 0x7f7bb40cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 >> v2:172.21.15.90:3300/0 conn(0x7f7bb40cb660 msgr2=0x7f7bb40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 --2- 172.21.15.83:0/3491762173 >> v2:172.21.15.90:3300/0 conn(0x7f7bb40cb660 0x7f7bb40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb41599d0 con 0x7f7bb415a180 2023-11-28T11:12:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f7bac0019f0 con 0x7f7bb415a180 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 >> v1:172.21.15.83:6789/0 conn(0x7f7bb415a180 legacy=0x7f7bb415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ba0003a00 0x7f7ba0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0004430 0x7f7ba00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 0x7f7ba0005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40cdfe0 con 0x7f7ba0003a00 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4159d60 con 0x7f7ba0004430 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.830+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba000ba10 con 0x7f7bb40cb290 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ba0003a00 0x7f7ba0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bb9ab1700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0004430 0x7f7ba00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bbaab3700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 0x7f7ba0005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 -- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 msgr2=0x7f7ba0005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb40cb290 0x7f7ba0005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 -- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0004430 msgr2=0x7f7ba00048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0004430 0x7f7ba00048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 -- 172.21.15.83:0/3491762173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba00035b0 con 0x7f7ba0003a00 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ba0003a00 0x7f7ba0003e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009b90 tx=0x7f7ba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9192cc90331f327d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000e6c0 con 0x7f7ba0003a00 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb000e820 con 0x7f7ba0003a00 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.831+0000 7f7bb92b0700 1 -- 172.21.15.83:0/3491762173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000eb90 con 0x7f7ba0003a00 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbc516700 1 -- 172.21.15.83:0/3491762173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ba0003a00 msgr2=0x7f7ba0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbc516700 1 --2- 172.21.15.83:0/3491762173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ba0003a00 0x7f7ba0003e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009b90 tx=0x7f7ba0005880 comp rx=0 tx=0).stop 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbaab3700 1 -- 172.21.15.83:0/3491762173 reap_dead start 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbc516700 1 -- 172.21.15.83:0/3491762173 shutdown_connections 2023-11-28T11:12:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbc516700 1 -- 172.21.15.83:0/3491762173 >> 172.21.15.83:0/3491762173 conn(0x7f7bb40cf4e0 msgr2=0x7f7bb40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbc516700 1 -- 172.21.15.83:0/3491762173 shutdown_connections 2023-11-28T11:12:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.832+0000 7f7bbc516700 1 -- 172.21.15.83:0/3491762173 wait complete. 2023-11-28T11:12:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.833+0000 7f7bbc516700 1 Processor -- start 2023-11-28T11:12:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.833+0000 7f7bbc516700 1 -- start start 2023-11-28T11:12:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbc516700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0003a00 0x7f7bb40aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbc516700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb41e2e30 0x7f7bb41ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbc516700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 0x7f7bb4156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbc516700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40c3010 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbc516700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40afb50 con 0x7f7ba0003a00 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbc516700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40af2b0 con 0x7f7bb41e2e30 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 0x7f7bb4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 0x7f7bb4156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53112/0 (socket says 172.21.15.83:53112) 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 -- 172.21.15.83:0/2629513029 learned_addr learned my addr 172.21.15.83:0/2629513029 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bb9ab1700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb41e2e30 0x7f7bb41ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0003a00 0x7f7bb40aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 -- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb41e2e30 msgr2=0x7f7bb41ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb41e2e30 0x7f7bb41ee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 -- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0003a00 msgr2=0x7f7bb40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0003a00 0x7f7bb40aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.834+0000 7f7bbaab3700 1 -- 172.21.15.83:0/2629513029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40cda20 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.835+0000 7f7bbaab3700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 0x7f7bb4156200 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7bac001330 tx=0x7f7bac00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.835+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00adb0 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.835+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac0133f0 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.835+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb40cd740 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.835+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0137a0 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.835+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb40cdd00 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.836+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb4074320 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.836+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7bac013900 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.837+0000 7f7baaffd700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7b980682f0 0x7f7b9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.837+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7bac0866b0 con 0x7f7bb41ef7c0 2023-11-28T11:12:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.837+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7b980682f0 0x7f7b9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.837+0000 7f7bba2b2700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7b980682f0 0x7f7b9806a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00090b0 tx=0x7f7bb001f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:07.842+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7bac052dd0 con 0x7f7bb41ef7c0 2023-11-28T11:12:08.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.069+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb40cab40 con 0x7f7bb41ef7c0 2023-11-28T11:12:08.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.069+0000 7f7baaffd700 1 -- 172.21.15.83:0/2629513029 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f7bac0563f0 con 0x7f7bb41ef7c0 2023-11-28T11:12:08.070 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:40.850456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:08.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7b980682f0 msgr2=0x7f7b9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7b980682f0 0x7f7b9806a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00090b0 tx=0x7f7bb001f040 comp rx=0 tx=0).stop 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 msgr2=0x7f7bb4156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 0x7f7bb4156200 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7bac001330 tx=0x7f7bac00a640 comp rx=0 tx=0).stop 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 shutdown_connections 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7b980682f0 0x7f7b9806a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bb41ef7c0 0x7f7bb4156200 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba0003a00 0x7f7bb40aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 --2- 172.21.15.83:0/2629513029 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bb41e2e30 0x7f7bb41ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 >> 172.21.15.83:0/2629513029 conn(0x7f7bb40cf4e0 msgr2=0x7f7bb4158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 shutdown_connections 2023-11-28T11:12:08.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.071+0000 7f7bbc516700 1 -- 172.21.15.83:0/2629513029 wait complete. 2023-11-28T11:12:08.073 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2023-11-28T11:12:08.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-11-28T11:12:08.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.204+0000 7f4711619700 1 Processor -- start 2023-11-28T11:12:08.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.204+0000 7f4711619700 1 -- start start 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f4711619700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c0cd880 0x7f470c0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f4711619700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f470c0b0240 0x7f470c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f4711619700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c06c410 0x7f470c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f4711619700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c06f9a0 con 0x7f470c06c410 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f4711619700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c06fc80 con 0x7f470c0b0240 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f4711619700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c06ff60 con 0x7f470c0cd880 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f470b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c06c410 0x7f470c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f470b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c06c410 0x7f470c1608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53118/0 (socket says 172.21.15.83:53118) 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.205+0000 7f470b7fe700 1 -- 172.21.15.83:0/175076119 learned_addr learned my addr 172.21.15.83:0/175076119 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470a7fc700 1 --2- 172.21.15.83:0/175076119 >> v2:172.21.15.90:3300/0 conn(0x7f470c0b0240 0x7f470c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470affd700 1 --2- 172.21.15.83:0/175076119 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c0cd880 0x7f470c0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470b7fe700 1 -- 172.21.15.83:0/175076119 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c0cd880 msgr2=0x7f470c0d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470b7fe700 1 --2- 172.21.15.83:0/175076119 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c0cd880 0x7f470c0d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470b7fe700 1 -- 172.21.15.83:0/175076119 >> v2:172.21.15.90:3300/0 conn(0x7f470c0b0240 msgr2=0x7f470c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470b7fe700 1 --2- 172.21.15.83:0/175076119 >> v2:172.21.15.90:3300/0 conn(0x7f470c0b0240 0x7f470c06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470b7fe700 1 -- 172.21.15.83:0/175076119 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c1603b0 con 0x7f470c06c410 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f470b7fe700 1 --2- 172.21.15.83:0/175076119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c06c410 0x7f470c1608b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f470c0d1df0 tx=0x7f470400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff589a28269ab8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f4709ffb700 1 -- 172.21.15.83:0/175076119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400e070 con 0x7f470c06c410 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f4709ffb700 1 -- 172.21.15.83:0/175076119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400b9c0 con 0x7f470c06c410 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.206+0000 7f4709ffb700 1 -- 172.21.15.83:0/175076119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400bd70 con 0x7f470c06c410 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 -- 172.21.15.83:0/175076119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c06c410 msgr2=0x7f470c1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 --2- 172.21.15.83:0/175076119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c06c410 0x7f470c1608b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f470c0d1df0 tx=0x7f470400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 -- 172.21.15.83:0/175076119 shutdown_connections 2023-11-28T11:12:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 --2- 172.21.15.83:0/175076119 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c0cd880 0x7f470c0d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 --2- 172.21.15.83:0/175076119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c06c410 0x7f470c1608b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 --2- 172.21.15.83:0/175076119 >> v2:172.21.15.90:3300/0 conn(0x7f470c0b0240 0x7f470c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 -- 172.21.15.83:0/175076119 >> 172.21.15.83:0/175076119 conn(0x7f470c06de20 msgr2=0x7f470c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 -- 172.21.15.83:0/175076119 shutdown_connections 2023-11-28T11:12:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.207+0000 7f4711619700 1 -- 172.21.15.83:0/175076119 wait complete. 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.208+0000 7f4711619700 1 Processor -- start 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.208+0000 7f4711619700 1 -- start start 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c06c410 0x7f470c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f470c0b0240 0x7f470c157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 0x7f470c1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c157e60 con 0x7f470c0cd880 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c1575c0 con 0x7f470c0b0240 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c15a900 con 0x7f470c06c410 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 0x7f470c1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 0x7f470c1f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53124/0 (socket says 172.21.15.83:53124) 2023-11-28T11:12:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 -- 172.21.15.83:0/3280548355 learned_addr learned my addr 172.21.15.83:0/3280548355 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470a7fc700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f470c0b0240 0x7f470c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470affd700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c06c410 0x7f470c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 -- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c06c410 msgr2=0x7f470c157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c06c410 0x7f470c157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 -- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f470c0b0240 msgr2=0x7f470c157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f470c0b0240 0x7f470c157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 -- 172.21.15.83:0/3280548355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704009b30 con 0x7f470c0cd880 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f470b7fe700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 0x7f470c1f1db0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f470400b730 tx=0x7f4704004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f46f37fe700 1 -- 172.21.15.83:0/3280548355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040099b0 con 0x7f470c0cd880 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f46f37fe700 1 -- 172.21.15.83:0/3280548355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704004250 con 0x7f470c0cd880 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c0cdc50 con 0x7f470c0cd880 2023-11-28T11:12:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.209+0000 7f46f37fe700 1 -- 172.21.15.83:0/3280548355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004600 con 0x7f470c0cd880 2023-11-28T11:12:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.210+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c06d670 con 0x7f470c0cd880 2023-11-28T11:12:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.211+0000 7f46f37fe700 1 -- 172.21.15.83:0/3280548355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4704004760 con 0x7f470c0cd880 2023-11-28T11:12:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.211+0000 7f46f37fe700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46f80682f0 0x7f46f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.211+0000 7f46f37fe700 1 -- 172.21.15.83:0/3280548355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f47040873c0 con 0x7f470c0cd880 2023-11-28T11:12:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.211+0000 7f470affd700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46f80682f0 0x7f46f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.212+0000 7f470affd700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46f80682f0 0x7f46f806a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4700006b20 tx=0x7f4700008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.213+0000 7f471d7a2b80 1 Processor -- start 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 -- start start 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 0x55f5d34c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d3506810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d33aac60 con 0x55f5d34ea8b0 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34193f0 con 0x55f5d34e81d0 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3412920 con 0x55f5d34eb1c0 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f470affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 0x55f5d34c0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f470affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 0x55f5d34c0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53128/0 (socket says 172.21.15.83:53128) 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f470affd700 1 -- 172.21.15.83:0/1983957669 learned_addr learned my addr 172.21.15.83:0/1983957669 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f470a7fc700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.214+0000 7f470b7fe700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d3506810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470affd700 1 -- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 msgr2=0x55f5d3506810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470affd700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d3506810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470affd700 1 -- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 msgr2=0x55f5d34eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470affd700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470affd700 1 -- 172.21.15.83:0/1983957669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d3294680 con 0x55f5d34ea8b0 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470a7fc700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470b7fe700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d3506810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f470affd700 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 0x55f5d34c0ac0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f47000109a0 tx=0x7f4700013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acdf48483aad5b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f46f0ff9700 1 -- 172.21.15.83:0/1983957669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700016900 con 0x55f5d34ea8b0 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f46f0ff9700 1 -- 172.21.15.83:0/1983957669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470001a070 con 0x55f5d34ea8b0 2023-11-28T11:12:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.215+0000 7f46f0ff9700 1 -- 172.21.15.83:0/1983957669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700016c60 con 0x55f5d34ea8b0 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 -- 172.21.15.83:0/1983957669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 msgr2=0x55f5d34c0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 0x55f5d34c0ac0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f47000109a0 tx=0x7f4700013000 comp rx=0 tx=0).stop 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 -- 172.21.15.83:0/1983957669 shutdown_connections 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d3506810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34ea8b0 0x55f5d34c0ac0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/1983957669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 -- 172.21.15.83:0/1983957669 >> 172.21.15.83:0/1983957669 conn(0x55f5d34bde00 msgr2=0x55f5d34be790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 -- 172.21.15.83:0/1983957669 shutdown_connections 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 -- 172.21.15.83:0/1983957669 wait complete. 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.216+0000 7f471d7a2b80 1 -- start start 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.217+0000 7f471d7a2b80 1 Processor -- start 2023-11-28T11:12:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34eb1c0 0x55f5d34ee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3296970 con 0x55f5d34e81d0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34dc340 con 0x55f5d34eb1c0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34df890 con 0x55f5d34ea8b0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 10 client.0 _ll_register_callbacks cb 0x55f5d32b82e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34fcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53134/0 (socket says 172.21.15.83:53134) 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 -- 172.21.15.83:0/370269466 learned_addr learned my addr 172.21.15.83:0/370269466 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470a7fc700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470b7fe700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34eb1c0 0x55f5d34ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 -- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 msgr2=0x55f5d34dee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34dee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 -- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34eb1c0 msgr2=0x55f5d34ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34eb1c0 0x55f5d34ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 -- 172.21.15.83:0/370269466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470000e040 con 0x55f5d34e81d0 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470b7fe700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34eb1c0 0x55f5d34ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f470affd700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34fcc10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x55f5d34bc170 tx=0x7f47000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb7569e7e5587cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f46e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f46e9ffb700 1 -- 172.21.15.83:0/370269466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700021890 con 0x55f5d34e81d0 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f46e9ffb700 1 -- 172.21.15.83:0/370269466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f47000219f0 con 0x55f5d34e81d0 2023-11-28T11:12:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.218+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5d33e01e0 con 0x55f5d34e81d0 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f46e9ffb700 1 -- 172.21.15.83:0/370269466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4700021bb0 con 0x55f5d34e81d0 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 10 client.6528 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55f5d3240e30 con 0x55f5d34e81d0 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f46e9ffb700 1 -- 172.21.15.83:0/370269466 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5d33e01e0 con 0x55f5d34e81d0 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f46e9ffb700 1 -- 172.21.15.83:0/370269466 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4700021d10 con 0x55f5d34e81d0 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 10 client.6528 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 10 client.6528 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 20 client.6528 populate_metadata read hostname 'smithi083' 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34db260 0x55f5d34db630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 4 client.6528 mds_command: new command op to 6231 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5d3296fb0 con 0x55f5d34db260 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f470a7fc700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34db260 0x55f5d34db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f46e9ffb700 10 client.6528 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.219+0000 7f470a7fc700 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34db260 0x55f5d34db630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5288c4fc585781bf server_cookie=2500a51206ab270 in_seq=0 out_seq=0 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f46e9ffb700 1 -- 172.21.15.83:0/370269466 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55f5d3296fb0 con 0x55f5d34db260 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f46e9ffb700 10 client.6528 handle_command_reply: tid=0 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 client.6528 shutdown 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 msgr2=0x55f5d34fcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34fcc10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x55f5d34bc170 tx=0x7f47000047c0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 shutdown_connections 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34db260 0x55f5d34db630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34eb1c0 0x55f5d34ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34fcc10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/370269466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f46e9ffb700 0 client.6528 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 >> 172.21.15.83:0/370269466 conn(0x55f5d34bde00 msgr2=0x55f5d34c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 shutdown_connections 2023-11-28T11:12:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 1 -- 172.21.15.83:0/370269466 wait complete. 2023-11-28T11:12:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.220+0000 7f471d7a2b80 20 client.6528 trim_cache size 0 max 16384 2023-11-28T11:12:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.233+0000 7f471d7a2b80 1 Processor -- start 2023-11-28T11:12:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.233+0000 7f471d7a2b80 1 -- start start 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 0x55f5d34fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34baab0 con 0x55f5d34eb1c0 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34b9a60 con 0x55f5d34e81d0 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34ba4a0 con 0x55f5d34ea8b0 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 0x55f5d34fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 0x55f5d34fc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53146/0 (socket says 172.21.15.83:53146) 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 -- 172.21.15.83:0/2515190296 learned_addr learned my addr 172.21.15.83:0/2515190296 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470a7fc700 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470affd700 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 -- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 msgr2=0x55f5d34bb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34bb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 -- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 msgr2=0x55f5d34fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 -- 172.21.15.83:0/2515190296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d33d91e0 con 0x55f5d34eb1c0 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470affd700 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f470b7fe700 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 0x55f5d34fc600 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4704053cd0 tx=0x7f4704087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee9ed0e242a19c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f46f17fa700 1 -- 172.21.15.83:0/2515190296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040870b0 con 0x55f5d34eb1c0 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f46f17fa700 1 -- 172.21.15.83:0/2515190296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704087210 con 0x55f5d34eb1c0 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.234+0000 7f46f17fa700 1 -- 172.21.15.83:0/2515190296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470405d0d0 con 0x55f5d34eb1c0 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 -- 172.21.15.83:0/2515190296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 msgr2=0x55f5d34fc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 0x55f5d34fc600 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4704053cd0 tx=0x7f4704087040 comp rx=0 tx=0).stop 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 -- 172.21.15.83:0/2515190296 shutdown_connections 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34eb1c0 0x55f5d34fc600 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34e81d0 0x55f5d34fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/2515190296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34ea8b0 0x55f5d34bb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 -- 172.21.15.83:0/2515190296 >> 172.21.15.83:0/2515190296 conn(0x55f5d34bde00 msgr2=0x55f5d34c1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 -- 172.21.15.83:0/2515190296 shutdown_connections 2023-11-28T11:12:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 -- 172.21.15.83:0/2515190296 wait complete. 2023-11-28T11:12:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 -- start start 2023-11-28T11:12:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.235+0000 7f471d7a2b80 1 Processor -- start 2023-11-28T11:12:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34ea8b0 0x55f5d34fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d34fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34de8e0 con 0x55f5d34e81d0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34c0850 con 0x55f5d34ea8b0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d34c0aa0 con 0x55f5d34eb1c0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f471d7a2b80 10 client.0 _ll_register_callbacks cb 0x55f5d32b82e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34e3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53148/0 (socket says 172.21.15.83:53148) 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470affd700 1 -- 172.21.15.83:0/3033162783 learned_addr learned my addr 172.21.15.83:0/3033162783 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470a7fc700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34ea8b0 0x55f5d34fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470b7fe700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d34fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470affd700 1 -- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 msgr2=0x55f5d34fc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.236+0000 7f470affd700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d34fc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f470affd700 1 -- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34ea8b0 msgr2=0x55f5d34fc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f470affd700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34ea8b0 0x55f5d34fc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f470affd700 1 -- 172.21.15.83:0/3033162783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c06d670 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f470b7fe700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d34fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f470affd700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34e3640 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f47000109a0 tx=0x7f4700016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f116617ba45e29a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f46e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f46e9ffb700 1 -- 172.21.15.83:0/3033162783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700046070 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f46e9ffb700 1 -- 172.21.15.83:0/3033162783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4700034580 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5d33d0200 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f46e9ffb700 1 -- 172.21.15.83:0/3033162783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4700006b50 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 10 client.6534 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55f5d32672a0 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f46e9ffb700 1 -- 172.21.15.83:0/3033162783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5d33d0200 con 0x55f5d34e81d0 2023-11-28T11:12:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f46e9ffb700 1 -- 172.21.15.83:0/3033162783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4700049020 con 0x55f5d34e81d0 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 10 client.6534 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 10 client.6534 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 20 client.6534 populate_metadata read hostname 'smithi083' 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34dd7d0 0x55f5d34ddba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 4 client.6534 mds_command: new command op to 6231 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 --> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55f5d33ba210 con 0x55f5d34dd7d0 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.237+0000 7f470a7fc700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34dd7d0 0x55f5d34ddba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f46e9ffb700 10 client.6534 ms_handle_connect on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f470a7fc700 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34dd7d0 0x55f5d34ddba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a3f65f467be749 server_cookie=2ec95a5f8f73b165 in_seq=0 out_seq=0 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f46e9ffb700 1 -- 172.21.15.83:0/3033162783 <== mds.0 v2:172.21.15.83:6835/3038492821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55f5d33ba210 con 0x55f5d34dd7d0 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f46e9ffb700 10 client.6534 handle_command_reply: tid=0 2023-11-28T11:12:08.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 client.6534 shutdown 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 msgr2=0x55f5d34e3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34e3640 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f47000109a0 tx=0x7f4700016000 comp rx=0 tx=0).stop 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 shutdown_connections 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:6835/3038492821,v1:172.21.15.83:6837/3038492821] conn(0x55f5d34dd7d0 0x55f5d34ddba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x55f5d34eb1c0 0x55f5d34fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f46e9ffb700 0 client.6534 ms_handle_reset on v2:172.21.15.83:6835/3038492821 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d34e81d0 0x55f5d34e3640 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 --2- 172.21.15.83:0/3033162783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f5d34ea8b0 0x55f5d34fc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 >> 172.21.15.83:0/3033162783 conn(0x55f5d34bde00 msgr2=0x55f5d34e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 shutdown_connections 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 1 -- 172.21.15.83:0/3033162783 wait complete. 2023-11-28T11:12:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f471d7a2b80 20 client.6534 trim_cache size 0 max 16384 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 2203794046, 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir_test/file" 2023-11-28T11:12:08.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-28T11:12:08.250 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-28T11:12:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46f80682f0 msgr2=0x7f46f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f4711619700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46f80682f0 0x7f46f806a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4700006b20 tx=0x7f4700008040 comp rx=0 tx=0).stop 2023-11-28T11:12:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 msgr2=0x7f470c1f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.238+0000 7f4711619700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 0x7f470c1f1db0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f470400b730 tx=0x7f4704004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 shutdown_connections 2023-11-28T11:12:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f46f80682f0 0x7f46f806a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0cd880 0x7f470c1f1db0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f470c06c410 0x7f470c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 --2- 172.21.15.83:0/3280548355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f470c0b0240 0x7f470c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 >> 172.21.15.83:0/3280548355 conn(0x7f470c06de20 msgr2=0x7f470c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 shutdown_connections 2023-11-28T11:12:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.239+0000 7f4711619700 1 -- 172.21.15.83:0/3280548355 wait complete. 2023-11-28T11:12:08.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:08.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.373+0000 7fd167aa2700 1 Processor -- start 2023-11-28T11:12:08.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.373+0000 7fd167aa2700 1 -- start start 2023-11-28T11:12:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd167aa2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd16006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd167aa2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd1600b18a0 0x7fd1600b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd167aa2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1600cf540 con 0x7fd160159830 2023-11-28T11:12:08.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd167aa2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16015d480 con 0x7fd1600b18a0 2023-11-28T11:12:08.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd167aa2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600b2260 con 0x7fd1600cf0d0 2023-11-28T11:12:08.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd16603f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd1600b18a0 0x7fd1600b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd16603f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd1600b18a0 0x7fd1600b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53158/0 (socket says 172.21.15.83:53158) 2023-11-28T11:12:08.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd16583e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd160159830 0x7fd16015de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51016/0 (socket says 172.21.15.83:51016) 2023-11-28T11:12:08.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.374+0000 7fd16603f700 1 -- 172.21.15.83:0/30953487 learned_addr learned my addr 172.21.15.83:0/30953487 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16503d700 1 --2- 172.21.15.83:0/30953487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd16006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16603f700 1 -- 172.21.15.83:0/30953487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 msgr2=0x7fd16006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16603f700 1 --2- 172.21.15.83:0/30953487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd16006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16603f700 1 -- 172.21.15.83:0/30953487 >> v1:172.21.15.83:6789/0 conn(0x7fd160159830 legacy=0x7fd16015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16483c700 1 -- 172.21.15.83:0/30953487 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475385745 0 0) 0x7fd1600cf540 con 0x7fd160159830 2023-11-28T11:12:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16603f700 1 -- 172.21.15.83:0/30953487 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1600cf790 con 0x7fd1600b18a0 2023-11-28T11:12:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16603f700 1 --2- 172.21.15.83:0/30953487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1600b18a0 0x7fd1600b1d20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd158009e50 tx=0x7fd15800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de480b629eaa681d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16483c700 1 -- 172.21.15.83:0/30953487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800e070 con 0x7fd1600b18a0 2023-11-28T11:12:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16483c700 1 -- 172.21.15.83:0/30953487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15800b9c0 con 0x7fd1600b18a0 2023-11-28T11:12:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd16483c700 1 -- 172.21.15.83:0/30953487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800bd70 con 0x7fd1600b18a0 2023-11-28T11:12:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd167aa2700 1 -- 172.21.15.83:0/30953487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1600b18a0 msgr2=0x7fd1600b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.375+0000 7fd167aa2700 1 --2- 172.21.15.83:0/30953487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1600b18a0 0x7fd1600b1d20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd158009e50 tx=0x7fd15800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.376+0000 7fd167aa2700 1 -- 172.21.15.83:0/30953487 shutdown_connections 2023-11-28T11:12:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.376+0000 7fd167aa2700 1 --2- 172.21.15.83:0/30953487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1600b18a0 0x7fd1600b1d20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.376+0000 7fd167aa2700 1 --2- 172.21.15.83:0/30953487 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd16006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.376+0000 7fd167aa2700 1 -- 172.21.15.83:0/30953487 >> 172.21.15.83:0/30953487 conn(0x7fd16006e9a0 msgr2=0x7fd1600cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.376+0000 7fd167aa2700 1 -- 172.21.15.83:0/30953487 shutdown_connections 2023-11-28T11:12:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.376+0000 7fd167aa2700 1 -- 172.21.15.83:0/30953487 wait complete. 2023-11-28T11:12:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.377+0000 7fd167aa2700 1 Processor -- start 2023-11-28T11:12:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.377+0000 7fd167aa2700 1 -- start start 2023-11-28T11:12:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1600b18a0 0x7fd160147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd1600c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 0x7fd1600c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd160144c50 con 0x7fd160159830 2023-11-28T11:12:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600c78c0 con 0x7fd1600b18a0 2023-11-28T11:12:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600c7ba0 con 0x7fd1600cf0d0 2023-11-28T11:12:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 0x7fd1600c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 0x7fd1600c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53164/0 (socket says 172.21.15.83:53164) 2023-11-28T11:12:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 -- 172.21.15.83:0/1636574643 learned_addr learned my addr 172.21.15.83:0/1636574643 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16503d700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd1600c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16583e700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1600b18a0 0x7fd160147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 -- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 msgr2=0x7fd1600c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd1600c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 -- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1600b18a0 msgr2=0x7fd160147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1600b18a0 0x7fd160147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 -- 172.21.15.83:0/1636574643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158009b30 con 0x7fd160159830 2023-11-28T11:12:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16583e700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1600b18a0 0x7fd160147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd16603f700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 0x7fd1600c5f90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd16006dc40 tx=0x7fd158004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580099b0 con 0x7fd160159830 2023-11-28T11:12:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1580092e0 con 0x7fd160159830 2023-11-28T11:12:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16015d690 con 0x7fd160159830 2023-11-28T11:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15801a6c0 con 0x7fd160159830 2023-11-28T11:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.378+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1600cfa70 con 0x7fd160159830 2023-11-28T11:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.379+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16015d970 con 0x7fd160159830 2023-11-28T11:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.380+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd15801a820 con 0x7fd160159830 2023-11-28T11:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.380+0000 7fd1567fc700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1440682f0 0x7fd14406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.380+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd158089480 con 0x7fd160159830 2023-11-28T11:12:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.380+0000 7fd16583e700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1440682f0 0x7fd14406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.380+0000 7fd16583e700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1440682f0 0x7fd14406a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd150000f60 tx=0x7fd150009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.384+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd158055ba0 con 0x7fd160159830 2023-11-28T11:12:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.608+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd160074320 con 0x7fd160159830 2023-11-28T11:12:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.608+0000 7fd1567fc700 1 -- 172.21.15.83:0/1636574643 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fd1580591c0 con 0x7fd160159830 2023-11-28T11:12:08.609 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:08.609 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1440682f0 msgr2=0x7fd14406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1440682f0 0x7fd14406a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd150000f60 tx=0x7fd150009040 comp rx=0 tx=0).stop 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 msgr2=0x7fd1600c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 0x7fd1600c5f90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd16006dc40 tx=0x7fd158004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 shutdown_connections 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd1440682f0 0x7fd14406a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd160159830 0x7fd1600c5f90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1600b18a0 0x7fd160147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 --2- 172.21.15.83:0/1636574643 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1600cf0d0 0x7fd1600c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 >> 172.21.15.83:0/1636574643 conn(0x7fd16006e9a0 msgr2=0x7fd1600ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 shutdown_connections 2023-11-28T11:12:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:08.610+0000 7fd167aa2700 1 -- 172.21.15.83:0/1636574643 wait complete. 2023-11-28T11:12:08.622 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:12:10.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:10.136 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:13.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.753+0000 7f757bd0b700 1 Processor -- start 2023-11-28T11:12:13.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.753+0000 7f757bd0b700 1 -- start start 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757bd0b700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7574159830 0x7f75740cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757bd0b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75740b18a0 0x7f75740b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757bd0b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75740b2220 con 0x7f757406c5f0 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757bd0b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757415d480 con 0x7f75740b18a0 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757bd0b700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757406ca60 con 0x7f7574159830 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75740b18a0 0x7f75740b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75740b18a0 0x7f75740b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53172/0 (socket says 172.21.15.83:53172) 2023-11-28T11:12:13.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 -- 172.21.15.83:0/2848047221 learned_addr learned my addr 172.21.15.83:0/2848047221 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f7579aa7700 1 --2- 172.21.15.83:0/2848047221 >> v2:172.21.15.90:3300/0 conn(0x7f7574159830 0x7f75740cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 -- 172.21.15.83:0/2848047221 >> v2:172.21.15.90:3300/0 conn(0x7f7574159830 msgr2=0x7f75740cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 --2- 172.21.15.83:0/2848047221 >> v2:172.21.15.90:3300/0 conn(0x7f7574159830 0x7f75740cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 -- 172.21.15.83:0/2848047221 >> v1:172.21.15.90:6789/0 conn(0x7f757406c5f0 legacy=0x7f757415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.754+0000 7f757a2a8700 1 -- 172.21.15.83:0/2848047221 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740cf790 con 0x7f75740b18a0 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757a2a8700 1 --2- 172.21.15.83:0/2848047221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f75740b1ce0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f757406dcb0 tx=0x7f756c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9584851b886df15c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f7578aa5700 1 -- 172.21.15.83:0/2848047221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00e070 con 0x7f75740b18a0 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f7578aa5700 1 -- 172.21.15.83:0/2848047221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00b910 con 0x7f75740b18a0 2023-11-28T11:12:13.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f7578aa5700 1 -- 172.21.15.83:0/2848047221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00bcc0 con 0x7f75740b18a0 2023-11-28T11:12:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 -- 172.21.15.83:0/2848047221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 msgr2=0x7f75740b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 --2- 172.21.15.83:0/2848047221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f75740b1ce0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f757406dcb0 tx=0x7f756c00b330 comp rx=0 tx=0).stop 2023-11-28T11:12:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 -- 172.21.15.83:0/2848047221 shutdown_connections 2023-11-28T11:12:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 --2- 172.21.15.83:0/2848047221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f75740b1ce0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 --2- 172.21.15.83:0/2848047221 >> v2:172.21.15.90:3300/0 conn(0x7f7574159830 0x7f75740cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 -- 172.21.15.83:0/2848047221 >> 172.21.15.83:0/2848047221 conn(0x7f757406e9a0 msgr2=0x7f75740cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 -- 172.21.15.83:0/2848047221 shutdown_connections 2023-11-28T11:12:13.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.755+0000 7f757bd0b700 1 -- 172.21.15.83:0/2848047221 wait complete. 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.756+0000 7f757bd0b700 1 Processor -- start 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.756+0000 7f757bd0b700 1 -- start start 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757bd0b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f757406c5f0 0x7f7574157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757bd0b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f757415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757bd0b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7574159830 0x7f757415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757bd0b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415abb0 con 0x7f75740b18a0 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757bd0b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415ae90 con 0x7f7574159830 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757bd0b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757415b170 con 0x7f757406c5f0 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f75792a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f757415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f75792a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f757415c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53176/0 (socket says 172.21.15.83:53176) 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f75792a6700 1 -- 172.21.15.83:0/4210547240 learned_addr learned my addr 172.21.15.83:0/4210547240 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f7579aa7700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f757406c5f0 0x7f7574157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.757+0000 7f757a2a8700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7574159830 0x7f757415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f75792a6700 1 -- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f757406c5f0 msgr2=0x7f7574157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f75792a6700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f757406c5f0 0x7f7574157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f75792a6700 1 -- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7574159830 msgr2=0x7f757415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f75792a6700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7574159830 0x7f757415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f75792a6700 1 -- 172.21.15.83:0/4210547240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009b30 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f75792a6700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f757415c0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f756400c4e0 tx=0x7f756400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640052d0 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564005430 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757415d690 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640038c0 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.758+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75740cfa70 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.759+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757415d970 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.759+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7564018020 con 0x7f75740b18a0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.760+0000 7f756a7fc700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75580682f0 0x7f755806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.760+0000 7f7579aa7700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75580682f0 0x7f755806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.760+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7564087940 con 0x7f75740b18a0 2023-11-28T11:12:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.760+0000 7f7579aa7700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75580682f0 0x7f755806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7570009b40 tx=0x7f7570009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.764+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7564054060 con 0x7f75740b18a0 2023-11-28T11:12:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.989+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f757415acf0 con 0x7f75740b18a0 2023-11-28T11:12:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.990+0000 7f756a7fc700 1 -- 172.21.15.83:0/4210547240 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f757415acf0 con 0x7f75740b18a0 2023-11-28T11:12:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:13.992 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75580682f0 msgr2=0x7f755806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75580682f0 0x7f755806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7570009b40 tx=0x7f7570009210 comp rx=0 tx=0).stop 2023-11-28T11:12:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 msgr2=0x7f757415c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f757415c0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f756400c4e0 tx=0x7f756400c7f0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 shutdown_connections 2023-11-28T11:12:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7574159830 0x7f757415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f757406c5f0 0x7f7574157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75580682f0 0x7f755806a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 --2- 172.21.15.83:0/4210547240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b18a0 0x7f757415c0a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 >> 172.21.15.83:0/4210547240 conn(0x7f757406e9a0 msgr2=0x7f75740cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 shutdown_connections 2023-11-28T11:12:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:13.992+0000 7f757bd0b700 1 -- 172.21.15.83:0/4210547240 wait complete. 2023-11-28T11:12:14.005 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:12:19.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:19.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.138+0000 7f796a348700 1 Processor -- start 2023-11-28T11:12:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f796a348700 1 -- start start 2023-11-28T11:12:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f796a348700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640ca680 0x7f796406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f796a348700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7964160410 0x7f79640ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:19.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f796a348700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796406cab0 con 0x7f79640b1b10 2023-11-28T11:12:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f796a348700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79640cf860 con 0x7f7964160410 2023-11-28T11:12:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f796a348700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796406b920 con 0x7f79640ca680 2023-11-28T11:12:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.139+0000 7f79688e5700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7964160410 0x7f79640ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79637fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640ca680 0x7f796406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f7963fff700 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f79640b1b10 0x7f796415fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:49388/0 (socket says 172.21.15.83:49388) 2023-11-28T11:12:19.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f7963fff700 1 -- 172.21.15.83:0/1810267147 learned_addr learned my addr 172.21.15.83:0/1810267147 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79637fe700 1 -- 172.21.15.83:0/1810267147 >> v2:172.21.15.90:3300/0 conn(0x7f7964160410 msgr2=0x7f79640ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79637fe700 1 --2- 172.21.15.83:0/1810267147 >> v2:172.21.15.90:3300/0 conn(0x7f7964160410 0x7f79640ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79637fe700 1 -- 172.21.15.83:0/1810267147 >> v1:172.21.15.90:6789/0 conn(0x7f79640b1b10 legacy=0x7f796415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79637fe700 1 -- 172.21.15.83:0/1810267147 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640cab20 con 0x7f79640ca680 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79688e5700 1 --2- 172.21.15.83:0/1810267147 >> v2:172.21.15.90:3300/0 conn(0x7f7964160410 0x7f79640ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f79637fe700 1 --2- 172.21.15.83:0/1810267147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640ca680 0x7f796406c570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7958009840 tx=0x7f7958009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef5934add5cb88f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f7962ffd700 1 -- 172.21.15.83:0/1810267147 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958011830 con 0x7f79640ca680 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f7962ffd700 1 -- 172.21.15.83:0/1810267147 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958011990 con 0x7f79640ca680 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.140+0000 7f7962ffd700 1 -- 172.21.15.83:0/1810267147 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958011d20 con 0x7f79640ca680 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 -- 172.21.15.83:0/1810267147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640ca680 msgr2=0x7f796406c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 --2- 172.21.15.83:0/1810267147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640ca680 0x7f796406c570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7958009840 tx=0x7f7958009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 -- 172.21.15.83:0/1810267147 shutdown_connections 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 --2- 172.21.15.83:0/1810267147 >> v2:172.21.15.90:3300/0 conn(0x7f7964160410 0x7f79640ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 --2- 172.21.15.83:0/1810267147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640ca680 0x7f796406c570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 -- 172.21.15.83:0/1810267147 >> 172.21.15.83:0/1810267147 conn(0x7f796406e9a0 msgr2=0x7f79640cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:19.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 -- 172.21.15.83:0/1810267147 shutdown_connections 2023-11-28T11:12:19.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.141+0000 7f796a348700 1 -- 172.21.15.83:0/1810267147 wait complete. 2023-11-28T11:12:19.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.142+0000 7f796a348700 1 Processor -- start 2023-11-28T11:12:19.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.142+0000 7f796a348700 1 -- start start 2023-11-28T11:12:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f796a348700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640b1b10 0x7f796415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f796a348700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79640ca680 0x7f79640ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f796a348700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 0x7f79640adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f796a348700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640ac280 con 0x7f7964160410 2023-11-28T11:12:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f796a348700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c6290 con 0x7f79640ca680 2023-11-28T11:12:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f796a348700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79640ab950 con 0x7f79640b1b10 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f79688e5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 0x7f79640adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f79688e5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 0x7f79640adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53192/0 (socket says 172.21.15.83:53192) 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f79688e5700 1 -- 172.21.15.83:0/1919936385 learned_addr learned my addr 172.21.15.83:0/1919936385 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.143+0000 7f79637fe700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79640ca680 0x7f79640ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f7963fff700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640b1b10 0x7f796415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f79688e5700 1 -- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640b1b10 msgr2=0x7f796415f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f79688e5700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640b1b10 0x7f796415f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f79688e5700 1 -- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79640ca680 msgr2=0x7f79640ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f79688e5700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79640ca680 0x7f79640ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f79688e5700 1 -- 172.21.15.83:0/1919936385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c009a70 con 0x7f7964160410 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f79688e5700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 0x7f79640adf90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f795c00efc0 tx=0x7f795c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00ca90 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c00f040 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79580094a0 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00ce30 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.144+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79640cb0e0 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.145+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79640cae00 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.145+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f795c014480 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.146+0000 7f7960ff9700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79500682f0 0x7f795006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.146+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f795c086560 con 0x7f7964160410 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.146+0000 7f7963fff700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79500682f0 0x7f795006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.146+0000 7f7963fff700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79500682f0 0x7f795006a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7954005910 tx=0x7f7954009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.150+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f795c052c20 con 0x7f7964160410 2023-11-28T11:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.373+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f79640cb3c0 con 0x7f7964160410 2023-11-28T11:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.374+0000 7f7960ff9700 1 -- 172.21.15.83:0/1919936385 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f795c009cd0 con 0x7f7964160410 2023-11-28T11:12:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:19.375 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79500682f0 msgr2=0x7f795006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79500682f0 0x7f795006a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7954005910 tx=0x7f7954009040 comp rx=0 tx=0).stop 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 msgr2=0x7f79640adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 0x7f79640adf90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f795c00efc0 tx=0x7f795c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 shutdown_connections 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f79500682f0 0x7f795006a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7964160410 0x7f79640adf90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79640b1b10 0x7f796415f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 --2- 172.21.15.83:0/1919936385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79640ca680 0x7f79640ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 >> 172.21.15.83:0/1919936385 conn(0x7f796406e9a0 msgr2=0x7f79640c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 shutdown_connections 2023-11-28T11:12:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:19.376+0000 7f796a348700 1 -- 172.21.15.83:0/1919936385 wait complete. 2023-11-28T11:12:19.389 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:12:24.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.521+0000 7ff1a8abf700 1 Processor -- start 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.521+0000 7ff1a8abf700 1 -- start start 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a8abf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1a415cc20 0x7ff1a406c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a8abf700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a4161450 con 0x7ff1a415c850 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a8abf700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a41608d0 con 0x7ff1a40cf1c0 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a8abf700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a40ce630 con 0x7ff1a415cc20 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a2d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1a415cc20 0x7ff1a406c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff1a415c850 0x7ff1a40ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51056/0 (socket says 172.21.15.83:51056) 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a259c700 1 -- 172.21.15.83:0/243497112 learned_addr learned my addr 172.21.15.83:0/243497112 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/243497112 >> v1:172.21.15.90:6789/0 conn(0x7ff1a40cf1c0 legacy=0x7ff1a406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/243497112 >> v1:172.21.15.83:6789/0 conn(0x7ff1a415c850 legacy=0x7ff1a40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/243497112 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4160bb0 con 0x7ff1a415cc20 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a159a700 1 -- 172.21.15.83:0/243497112 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964737063 0 0) 0x7ff1a4161450 con 0x7ff1a415c850 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a2d9d700 1 --2- 172.21.15.83:0/243497112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a406c690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009d90 tx=0x7ff19c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4f586e4733e30d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a159a700 1 -- 172.21.15.83:0/243497112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00e070 con 0x7ff1a415cc20 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a159a700 1 -- 172.21.15.83:0/243497112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c00bbd0 con 0x7ff1a415cc20 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.522+0000 7ff1a159a700 1 -- 172.21.15.83:0/243497112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c012430 con 0x7ff1a415cc20 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 -- 172.21.15.83:0/243497112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 msgr2=0x7ff1a406c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/243497112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a406c690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009d90 tx=0x7ff19c00b330 comp rx=0 tx=0).stop 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 -- 172.21.15.83:0/243497112 shutdown_connections 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/243497112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a406c690 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 -- 172.21.15.83:0/243497112 >> 172.21.15.83:0/243497112 conn(0x7ff1a40cd960 msgr2=0x7ff1a40cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 -- 172.21.15.83:0/243497112 shutdown_connections 2023-11-28T11:12:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.523+0000 7ff1a8abf700 1 -- 172.21.15.83:0/243497112 wait complete. 2023-11-28T11:12:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.524+0000 7ff1a8abf700 1 Processor -- start 2023-11-28T11:12:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.524+0000 7ff1a8abf700 1 -- start start 2023-11-28T11:12:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a8abf700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff1a40cf1c0 0x7ff1a40c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a8abf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a415c850 0x7ff1a40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a8abf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a8abf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40c48e0 con 0x7ff1a415cc20 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a8abf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40c4bc0 con 0x7ff1a415c850 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a8abf700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40c4ea0 con 0x7ff1a40cf1c0 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a40b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53206/0 (socket says 172.21.15.83:53206) 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.525+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/23009798 learned_addr learned my addr 172.21.15.83:0/23009798 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a1d9b700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a415c850 0x7ff1a40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a259c700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff1a40cf1c0 0x7ff1a40c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:24.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff1a40cf1c0 msgr2=0x7ff1a40c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a2d9d700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff1a40cf1c0 0x7ff1a40c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a415c850 msgr2=0x7ff1a40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a2d9d700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a415c850 0x7ff1a40c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a2d9d700 1 -- 172.21.15.83:0/23009798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c009a70 con 0x7ff1a415cc20 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a1d9b700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a415c850 0x7ff1a40c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a2d9d700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a40b6450 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff19c014040 tx=0x7ff19c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c01b810 con 0x7ff1a415cc20 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c020070 con 0x7ff1a415cc20 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a4161170 con 0x7ff1a415cc20 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c01bbb0 con 0x7ff1a415cc20 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.526+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a4160e90 con 0x7ff1a415cc20 2023-11-28T11:12:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.527+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4160560 con 0x7ff1a415cc20 2023-11-28T11:12:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.527+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff19c01bd10 con 0x7ff1a415cc20 2023-11-28T11:12:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.528+0000 7ff192ffd700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.528+0000 7ff1a259c700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.528+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff19c022070 con 0x7ff1a415cc20 2023-11-28T11:12:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.528+0000 7ff1a259c700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff1840682f0 0x7ff18406a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff194000f60 tx=0x7ff194009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:24.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.532+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff19c054d90 con 0x7ff1a415cc20 2023-11-28T11:12:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.756+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff1a4074320 con 0x7ff1a415cc20 2023-11-28T11:12:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.756+0000 7ff192ffd700 1 -- 172.21.15.83:0/23009798 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff19c0583b0 con 0x7ff1a415cc20 2023-11-28T11:12:24.758 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:24.758 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff1840682f0 msgr2=0x7ff18406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff1840682f0 0x7ff18406a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff194000f60 tx=0x7ff194009040 comp rx=0 tx=0).stop 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 msgr2=0x7ff1a40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a40b6450 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff19c014040 tx=0x7ff19c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 shutdown_connections 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1a415cc20 0x7ff1a40b6450 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff1a40cf1c0 0x7ff1a40c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 --2- 172.21.15.83:0/23009798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a415c850 0x7ff1a40c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 >> 172.21.15.83:0/23009798 conn(0x7ff1a40cd960 msgr2=0x7ff1a40b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 shutdown_connections 2023-11-28T11:12:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:24.758+0000 7ff1a8abf700 1 -- 172.21.15.83:0/23009798 wait complete. 2023-11-28T11:12:24.771 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:12:29.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:29.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.902+0000 7f907437c700 1 Processor -- start 2023-11-28T11:12:29.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.902+0000 7f907437c700 1 -- start start 2023-11-28T11:12:29.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.902+0000 7f907437c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f906c15cd10 0x7f906c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:29.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f907437c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0cbfb0 con 0x7f906c0b1fb0 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f907437c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0cb710 con 0x7f906c15d0e0 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f907437c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c0cb9f0 con 0x7f906c15cd10 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f906c15cd10 0x7f906c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f906c15cd10 0x7f906c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53210/0 (socket says 172.21.15.83:53210) 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072919700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f906c0b1fb0 0x7f906c06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51074/0 (socket says 172.21.15.83:51074) 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 -- 172.21.15.83:0/2658618363 learned_addr learned my addr 172.21.15.83:0/2658618363 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 -- 172.21.15.83:0/2658618363 >> v1:172.21.15.90:6789/0 conn(0x7f906c15d0e0 legacy=0x7f906c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 -- 172.21.15.83:0/2658618363 >> v1:172.21.15.83:6789/0 conn(0x7f906c0b1fb0 legacy=0x7f906c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 -- 172.21.15.83:0/2658618363 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0cbcd0 con 0x7f906c15cd10 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9071116700 1 -- 172.21.15.83:0/2658618363 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146281073 0 0) 0x7f906c0cbfb0 con 0x7f906c0b1fb0 2023-11-28T11:12:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9072118700 1 --2- 172.21.15.83:0/2658618363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15cd10 0x7f906c0ca970 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b81fec09b83e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9071116700 1 -- 172.21.15.83:0/2658618363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c004230 con 0x7f906c15cd10 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9071116700 1 -- 172.21.15.83:0/2658618363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c004390 con 0x7f906c15cd10 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.903+0000 7f9071116700 1 -- 172.21.15.83:0/2658618363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0036a0 con 0x7f906c15cd10 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 -- 172.21.15.83:0/2658618363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15cd10 msgr2=0x7f906c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 --2- 172.21.15.83:0/2658618363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15cd10 0x7f906c0ca970 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 -- 172.21.15.83:0/2658618363 shutdown_connections 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 --2- 172.21.15.83:0/2658618363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15cd10 0x7f906c0ca970 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 -- 172.21.15.83:0/2658618363 >> 172.21.15.83:0/2658618363 conn(0x7f906c06e9a0 msgr2=0x7f906c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 -- 172.21.15.83:0/2658618363 shutdown_connections 2023-11-28T11:12:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.904+0000 7f907437c700 1 -- 172.21.15.83:0/2658618363 wait complete. 2023-11-28T11:12:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.905+0000 7f907437c700 1 Processor -- start 2023-11-28T11:12:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.905+0000 7f907437c700 1 -- start start 2023-11-28T11:12:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f907437c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f906c0b1fb0 0x7f906c0b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f907437c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c15cd10 0x7f906c1479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f907437c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 0x7f906c0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f907437c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0b52e0 con 0x7f906c15d0e0 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f907437c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0b4430 con 0x7f906c15cd10 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f907437c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c147ee0 con 0x7f906c0b1fb0 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 0x7f906c0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 0x7f906c0c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53220/0 (socket says 172.21.15.83:53220) 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 -- 172.21.15.83:0/407532905 learned_addr learned my addr 172.21.15.83:0/407532905 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9071917700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c15cd10 0x7f906c1479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072118700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f906c0b1fb0 0x7f906c0b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 -- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f906c0b1fb0 msgr2=0x7f906c0b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f906c0b1fb0 0x7f906c0b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 -- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c15cd10 msgr2=0x7f906c1479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c15cd10 0x7f906c1479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9072919700 1 -- 172.21.15.83:0/407532905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c009550 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.906+0000 7f9071917700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c15cd10 0x7f906c1479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.907+0000 7f9072919700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 0x7f906c0c8a80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f906400cc10 tx=0x7f906400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.907+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906400abb0 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.907+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906400ad10 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.907+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c06b660 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.907+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064007310 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.907+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c06b9b0 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.908+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c0cd9f0 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.908+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f906401a030 con 0x7f906c15d0e0 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.908+0000 7f9062ffd700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90500682f0 0x7f905006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.909+0000 7f9072118700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90500682f0 0x7f905006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.909+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9064085bb0 con 0x7f906c15d0e0 2023-11-28T11:12:29.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.909+0000 7f9072118700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90500682f0 0x7f905006a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f905c009130 tx=0x7f905c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:29.913+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f906408c080 con 0x7f906c15d0e0 2023-11-28T11:12:30.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.138+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f906c074320 con 0x7f906c15d0e0 2023-11-28T11:12:30.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.138+0000 7f9062ffd700 1 -- 172.21.15.83:0/407532905 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f90640180c0 con 0x7f906c15d0e0 2023-11-28T11:12:30.140 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:30.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90500682f0 msgr2=0x7f905006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90500682f0 0x7f905006a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f905c009130 tx=0x7f905c008030 comp rx=0 tx=0).stop 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 msgr2=0x7f906c0c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 0x7f906c0c8a80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f906400cc10 tx=0x7f906400a060 comp rx=0 tx=0).stop 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 shutdown_connections 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f90500682f0 0x7f905006a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f906c15d0e0 0x7f906c0c8a80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f906c0b1fb0 0x7f906c0b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 --2- 172.21.15.83:0/407532905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c15cd10 0x7f906c1479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 >> 172.21.15.83:0/407532905 conn(0x7f906c06e9a0 msgr2=0x7f906c0caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:30.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 shutdown_connections 2023-11-28T11:12:30.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.140+0000 7f907437c700 1 -- 172.21.15.83:0/407532905 wait complete. 2023-11-28T11:12:30.153 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:12:30.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:12:30.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:12:30.242 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:12:30.243 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:12:30.243 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:30.261 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:12:30.261 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:30.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:30.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.478+0000 7f71b2bdf700 1 Processor -- start 2023-11-28T11:12:30.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.478+0000 7f71b2bdf700 1 -- start start 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.478+0000 7f71b2bdf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71ac15c850 0x7f71ac0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.478+0000 7f71b2bdf700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b2bdf700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac161450 con 0x7f71ac0d09f0 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b2bdf700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0ce630 con 0x7f71ac15c850 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b2bdf700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac1608d0 con 0x7f71ac15cc20 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71ac15c850 0x7f71ac0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71ac15c850 0x7f71ac0ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53224/0 (socket says 172.21.15.83:53224) 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 -- 172.21.15.83:0/1568593496 learned_addr learned my addr 172.21.15.83:0/1568593496 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 -- 172.21.15.83:0/1568593496 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 msgr2=0x7f71ac0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:30.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 --2- 172.21.15.83:0/1568593496 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 -- 172.21.15.83:0/1568593496 >> v1:172.21.15.90:6789/0 conn(0x7f71ac0d09f0 legacy=0x7f71ac06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 -- 172.21.15.83:0/1568593496 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac160bb0 con 0x7f71ac15c850 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71b097b700 1 --2- 172.21.15.83:0/1568593496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac15c850 0x7f71ac0ceab0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f71a00098f0 tx=0x7f71a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ed68019cdc41e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71ab7fe700 1 -- 172.21.15.83:0/1568593496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0004230 con 0x7f71ac15c850 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71ab7fe700 1 -- 172.21.15.83:0/1568593496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a0004390 con 0x7f71ac15c850 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.479+0000 7f71ab7fe700 1 -- 172.21.15.83:0/1568593496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00036a0 con 0x7f71ac15c850 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1568593496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac15c850 msgr2=0x7f71ac0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1568593496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac15c850 0x7f71ac0ceab0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f71a00098f0 tx=0x7f71a0009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1568593496 shutdown_connections 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1568593496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac15c850 0x7f71ac0ceab0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1568593496 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1568593496 >> 172.21.15.83:0/1568593496 conn(0x7f71ac0cd960 msgr2=0x7f71ac0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:30.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1568593496 shutdown_connections 2023-11-28T11:12:30.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.480+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1568593496 wait complete. 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.481+0000 7f71b2bdf700 1 Processor -- start 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.481+0000 7f71b2bdf700 1 -- start start 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b2bdf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 0x7f71ac15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b2bdf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71ac15c850 0x7f71ac0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b2bdf700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b2bdf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0c4f50 con 0x7f71ac0d09f0 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b2bdf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0c6400 con 0x7f71ac15c850 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b2bdf700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0c51b0 con 0x7f71ac15cc20 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 0x7f71ac15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 0x7f71ac15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53230/0 (socket says 172.21.15.83:53230) 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 -- 172.21.15.83:0/1345301915 learned_addr learned my addr 172.21.15.83:0/1345301915 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71abfff700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71ac15c850 0x7f71ac0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 -- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 msgr2=0x7f71ac15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b117c700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 -- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71ac15c850 msgr2=0x7f71ac0c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71ac15c850 0x7f71ac0c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 -- 172.21.15.83:0/1345301915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0009550 con 0x7f71ac0d09f0 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71abfff700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71ac15c850 0x7f71ac0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b117c700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.482+0000 7f71b097b700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 0x7f71ac15bce0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0d0e80 tx=0x7f71a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:30.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.483+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0003e70 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.483+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a001f070 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.483+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac161170 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.483+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000b8a0 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.483+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac160e90 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.484+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac160560 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.484+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f71a002a020 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.484+0000 7f71a97fa700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71900682f0 0x7f719006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.484+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f71a0087810 con 0x7f71ac0d09f0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.484+0000 7f71abfff700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71900682f0 0x7f719006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.485+0000 7f71abfff700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71900682f0 0x7f719006a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f71ac1f6930 tx=0x7f7198008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:30.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.489+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f71a0053ef0 con 0x7f71ac0d09f0 2023-11-28T11:12:30.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.672+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71ac160560 con 0x7f71ac0d09f0 2023-11-28T11:12:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.674+0000 7f71a97fa700 1 -- 172.21.15.83:0/1345301915 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18175 (secure 0 0 0) 0x7f71a0057510 con 0x7f71ac0d09f0 2023-11-28T11:12:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:30.675 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:12:25.948014+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71900682f0 msgr2=0x7f719006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71900682f0 0x7f719006a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f71ac1f6930 tx=0x7f7198008040 comp rx=0 tx=0).stop 2023-11-28T11:12:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 msgr2=0x7f71ac15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 0x7f71ac15bce0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0d0e80 tx=0x7f71a0004750 comp rx=0 tx=0).stop 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 shutdown_connections 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71ac15cc20 0x7f71ac15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f71900682f0 0x7f719006a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac0d09f0 0x7f71ac15bce0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 --2- 172.21.15.83:0/1345301915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71ac15c850 0x7f71ac0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 >> 172.21.15.83:0/1345301915 conn(0x7f71ac0cd960 msgr2=0x7f71ac06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 shutdown_connections 2023-11-28T11:12:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.676+0000 7f71b2bdf700 1 -- 172.21.15.83:0/1345301915 wait complete. 2023-11-28T11:12:30.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:11:22.194488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:11:23.354896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:12:30.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:12:30.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.811+0000 7fbd9bc75700 1 Processor -- start 2023-11-28T11:12:30.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.811+0000 7fbd9bc75700 1 -- start start 2023-11-28T11:12:30.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9bc75700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd940aee60 0x7fbd940b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9bc75700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9406d4a0 con 0x7fbd9415c890 2023-11-28T11:12:30.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9bc75700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940b5ae0 con 0x7fbd9406e370 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9bc75700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9406c400 con 0x7fbd940aee60 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9a212700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd940aee60 0x7fbd940b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd99a11700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd9415c890 0x7fbd9406dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51098/0 (socket says 172.21.15.83:51098) 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9a212700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd940aee60 0x7fbd940b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53242/0 (socket says 172.21.15.83:53242) 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd99a11700 1 -- 172.21.15.83:0/1208980615 learned_addr learned my addr 172.21.15.83:0/1208980615 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9a212700 1 -- 172.21.15.83:0/1208980615 >> v1:172.21.15.90:6789/0 conn(0x7fbd9406e370 legacy=0x7fbd9406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9a212700 1 -- 172.21.15.83:0/1208980615 >> v1:172.21.15.83:6789/0 conn(0x7fbd9415c890 legacy=0x7fbd9406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9a212700 1 -- 172.21.15.83:0/1208980615 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940b5240 con 0x7fbd940aee60 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd98a0f700 1 -- 172.21.15.83:0/1208980615 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728383966 0 0) 0x7fbd9406d4a0 con 0x7fbd9415c890 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd9a212700 1 --2- 172.21.15.83:0/1208980615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd940aee60 0x7fbd940b4bb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009d90 tx=0x7fbd8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116ed5dbc179b921 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd98a0f700 1 -- 172.21.15.83:0/1208980615 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00e070 con 0x7fbd940aee60 2023-11-28T11:12:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd98a0f700 1 -- 172.21.15.83:0/1208980615 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c00bb90 con 0x7fbd940aee60 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.812+0000 7fbd98a0f700 1 -- 172.21.15.83:0/1208980615 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c003a20 con 0x7fbd940aee60 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1208980615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd940aee60 msgr2=0x7fbd940b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1208980615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd940aee60 0x7fbd940b4bb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009d90 tx=0x7fbd8c00b330 comp rx=0 tx=0).stop 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1208980615 shutdown_connections 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1208980615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd940aee60 0x7fbd940b4bb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1208980615 >> 172.21.15.83:0/1208980615 conn(0x7fbd940b2b60 msgr2=0x7fbd940b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1208980615 shutdown_connections 2023-11-28T11:12:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.813+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1208980615 wait complete. 2023-11-28T11:12:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.814+0000 7fbd9bc75700 1 Processor -- start 2023-11-28T11:12:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.814+0000 7fbd9bc75700 1 -- start start 2023-11-28T11:12:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9bc75700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbd9406e370 0x7fbd94155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9bc75700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd940aee60 0x7fbd940c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9bc75700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 0x7fbd9415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9bc75700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94155fd0 con 0x7fbd9415c890 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9bc75700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94156410 con 0x7fbd940aee60 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9bc75700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940c3890 con 0x7fbd9406e370 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 0x7fbd9415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 0x7fbd9415cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53246/0 (socket says 172.21.15.83:53246) 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 -- 172.21.15.83:0/1220997365 learned_addr learned my addr 172.21.15.83:0/1220997365 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd99a11700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbd9406e370 0x7fbd94155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd99210700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd940aee60 0x7fbd940c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 -- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbd9406e370 msgr2=0x7fbd94155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbd9406e370 0x7fbd94155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 -- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd940aee60 msgr2=0x7fbd940c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd940aee60 0x7fbd940c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 -- 172.21.15.83:0/1220997365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009a70 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.815+0000 7fbd9a212700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 0x7fbd9415cda0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbd9408a420 tx=0x7fbd8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.816+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c012630 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.816+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c0127b0 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.816+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd940b5800 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.816+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c012b60 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.816+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd940b5520 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.817+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9406ac30 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.817+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd8c012cc0 con 0x7fbd9415c890 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.817+0000 7fbd8a7fc700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd780682f0 0x7fbd7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.817+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbd8c088330 con 0x7fbd9415c890 2023-11-28T11:12:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.817+0000 7fbd99a11700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd780682f0 0x7fbd7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.818+0000 7fbd99a11700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd780682f0 0x7fbd7806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbd840010e0 tx=0x7fbd84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:30.822+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd8c054920 con 0x7fbd9415c890 2023-11-28T11:12:31.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.002+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd940d0230 con 0x7fbd9415c890 2023-11-28T11:12:31.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.003+0000 7fbd8a7fc700 1 -- 172.21.15.83:0/1220997365 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd8c057f40 con 0x7fbd9415c890 2023-11-28T11:12:31.004 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:12:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd780682f0 msgr2=0x7fbd7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd780682f0 0x7fbd7806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbd840010e0 tx=0x7fbd84009040 comp rx=0 tx=0).stop 2023-11-28T11:12:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 msgr2=0x7fbd9415cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 0x7fbd9415cda0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbd9408a420 tx=0x7fbd8c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 shutdown_connections 2023-11-28T11:12:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd780682f0 0x7fbd7806a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9415c890 0x7fbd9415cda0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbd9406e370 0x7fbd94155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 --2- 172.21.15.83:0/1220997365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd940aee60 0x7fbd940c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 >> 172.21.15.83:0/1220997365 conn(0x7fbd940b2b60 msgr2=0x7fbd940b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:31.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 shutdown_connections 2023-11-28T11:12:31.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.005+0000 7fbd9bc75700 1 -- 172.21.15.83:0/1220997365 wait complete. 2023-11-28T11:12:31.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:12:31.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.139+0000 7fd35afa1700 1 Processor -- start 2023-11-28T11:12:31.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35afa1700 1 -- start start 2023-11-28T11:12:31.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35afa1700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd35415dee0 0x7fd3540ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35afa1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd35406d260 0x7fd3540cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35afa1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3540ce520 con 0x7fd35415db10 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35afa1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3540cd910 con 0x7fd35406d260 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35afa1700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3540cdbf0 con 0x7fd35415dee0 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd35953e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd35406d260 0x7fd3540cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.140+0000 7fd358d3d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd35415db10 0x7fd3540b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51112/0 (socket says 172.21.15.83:51112) 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd358d3d700 1 -- 172.21.15.83:0/3789588712 learned_addr learned my addr 172.21.15.83:0/3789588712 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd353fff700 1 --2- 172.21.15.83:0/3789588712 >> v2:172.21.15.90:3300/0 conn(0x7fd35415dee0 0x7fd3540ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd35953e700 1 -- 172.21.15.83:0/3789588712 >> v2:172.21.15.90:3300/0 conn(0x7fd35415dee0 msgr2=0x7fd3540ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd35953e700 1 --2- 172.21.15.83:0/3789588712 >> v2:172.21.15.90:3300/0 conn(0x7fd35415dee0 0x7fd3540ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd35953e700 1 -- 172.21.15.83:0/3789588712 >> v1:172.21.15.83:6789/0 conn(0x7fd35415db10 legacy=0x7fd3540b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd3537fe700 1 -- 172.21.15.83:0/3789588712 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787852953 0 0) 0x7fd3540ce520 con 0x7fd35415db10 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd35953e700 1 -- 172.21.15.83:0/3789588712 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540cded0 con 0x7fd35406d260 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd35953e700 1 --2- 172.21.15.83:0/3789588712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35406d260 0x7fd3540cfa00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd35415f4c0 tx=0x7fd34c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6849a3d3d4f6ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd3537fe700 1 -- 172.21.15.83:0/3789588712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00e070 con 0x7fd35406d260 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd3537fe700 1 -- 172.21.15.83:0/3789588712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c00bb20 con 0x7fd35406d260 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.141+0000 7fd3537fe700 1 -- 172.21.15.83:0/3789588712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c012430 con 0x7fd35406d260 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 -- 172.21.15.83:0/3789588712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35406d260 msgr2=0x7fd3540cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3789588712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35406d260 0x7fd3540cfa00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd35415f4c0 tx=0x7fd34c00b330 comp rx=0 tx=0).stop 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 -- 172.21.15.83:0/3789588712 shutdown_connections 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3789588712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35406d260 0x7fd3540cfa00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3789588712 >> v2:172.21.15.90:3300/0 conn(0x7fd35415dee0 0x7fd3540ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 -- 172.21.15.83:0/3789588712 >> 172.21.15.83:0/3789588712 conn(0x7fd3540b2b60 msgr2=0x7fd3540b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:31.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 -- 172.21.15.83:0/3789588712 shutdown_connections 2023-11-28T11:12:31.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.142+0000 7fd35afa1700 1 -- 172.21.15.83:0/3789588712 wait complete. 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.143+0000 7fd35afa1700 1 Processor -- start 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.143+0000 7fd35afa1700 1 -- start start 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35afa1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35406d260 0x7fd3540c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35afa1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd35415db10 0x7fd35406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35afa1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 0x7fd3540c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35afa1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540cbbd0 con 0x7fd35415dee0 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35afa1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540ca0a0 con 0x7fd35406d260 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35afa1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541598f0 con 0x7fd35415db10 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 0x7fd3540c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 0x7fd3540c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53262/0 (socket says 172.21.15.83:53262) 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 -- 172.21.15.83:0/3543685423 learned_addr learned my addr 172.21.15.83:0/3543685423 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd358d3d700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35406d260 0x7fd3540c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd353fff700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd35415db10 0x7fd35406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 -- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd35415db10 msgr2=0x7fd35406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd35415db10 0x7fd35406f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 -- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35406d260 msgr2=0x7fd3540c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35406d260 0x7fd3540c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 -- 172.21.15.83:0/3543685423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009a70 con 0x7fd35415dee0 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd35953e700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 0x7fd3540c2600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd35408a420 tx=0x7fd34c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.144+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c012430 con 0x7fd35415dee0 2023-11-28T11:12:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.145+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c020070 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.145+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c004280 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.145+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3540ceef0 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.145+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3540ce1b0 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.145+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3540cff40 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.146+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd34c02e070 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.146+0000 7fd3517fa700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd3380682f0 0x7fd33806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.146+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd34c022070 con 0x7fd35415dee0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.146+0000 7fd358d3d700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd3380682f0 0x7fd33806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.147+0000 7fd358d3d700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd3380682f0 0x7fd33806a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd3480010e0 tx=0x7fd348009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.151+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd34c054030 con 0x7fd35415dee0 2023-11-28T11:12:31.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.330+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3540742b0 con 0x7fd35415dee0 2023-11-28T11:12:31.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.331+0000 7fd3517fa700 1 -- 172.21.15.83:0/3543685423 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd34c057650 con 0x7fd35415dee0 2023-11-28T11:12:31.332 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:12:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd3380682f0 msgr2=0x7fd33806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd3380682f0 0x7fd33806a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd3480010e0 tx=0x7fd348009040 comp rx=0 tx=0).stop 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 msgr2=0x7fd3540c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 0x7fd3540c2600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd35408a420 tx=0x7fd34c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 shutdown_connections 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd3380682f0 0x7fd33806a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415dee0 0x7fd3540c2600 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd35406d260 0x7fd3540c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 --2- 172.21.15.83:0/3543685423 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd35415db10 0x7fd35406f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 >> 172.21.15.83:0/3543685423 conn(0x7fd3540b2b60 msgr2=0x7fd3540b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 shutdown_connections 2023-11-28T11:12:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.333+0000 7fd35afa1700 1 -- 172.21.15.83:0/3543685423 wait complete. 2023-11-28T11:12:31.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:12:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.468+0000 7fe931eff700 1 Processor -- start 2023-11-28T11:12:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.468+0000 7fe931eff700 1 -- start start 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe931eff700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe92c15c890 0x7fe92c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe931eff700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c06d4a0 con 0x7fe92c0b4bb0 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe931eff700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0b5ae0 con 0x7fe92c06e370 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe931eff700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0b5020 con 0x7fe92c15c890 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe92c0b4bb0 0x7fe92c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51128/0 (socket says 172.21.15.83:51128) 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92affd700 1 -- 172.21.15.83:0/3024266999 learned_addr learned my addr 172.21.15.83:0/3024266999 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92b7fe700 1 --2- 172.21.15.83:0/3024266999 >> v2:172.21.15.90:3300/0 conn(0x7fe92c15c890 0x7fe92c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904511939 0 0) 0x7fe92c06d4a0 con 0x7fe92c0b4bb0 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9140035b0 con 0x7fe92c0b4bb0 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1589600818 0 0) 0x7fe9140035b0 con 0x7fe92c0b4bb0 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 >> v2:172.21.15.90:3300/0 conn(0x7fe92c15c890 msgr2=0x7fe92c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 --2- 172.21.15.83:0/3024266999 >> v2:172.21.15.90:3300/0 conn(0x7fe92c15c890 0x7fe92c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 >> v1:172.21.15.90:6789/0 conn(0x7fe92c06e370 legacy=0x7fe92c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c06c400 con 0x7fe92c0b4bb0 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fe91c001670 con 0x7fe92c0b4bb0 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92b7fe700 1 --2- 172.21.15.83:0/3024266999 >> v2:172.21.15.90:3300/0 conn(0x7fe92c15c890 0x7fe92c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 >> v1:172.21.15.83:6789/0 conn(0x7fe92c0b4bb0 legacy=0x7fe92c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe914003a50 0x7fe914003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe914004850 0x7fe914004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe91400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b5020 con 0x7fe914005250 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91400c160 con 0x7fe914004850 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.469+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91400c320 con 0x7fe914003a50 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe91400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92affd700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe914004850 0x7fe914004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92b7fe700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe914003a50 0x7fe914003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 -- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe914003a50 msgr2=0x7fe914003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe914003a50 0x7fe914003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 -- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe914004850 msgr2=0x7fe914004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe914004850 0x7fe914004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 -- 172.21.15.83:0/3024266999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140035b0 con 0x7fe914005250 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 -- 172.21.15.83:0/3024266999 reap_dead start 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92bfff700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe91400ba10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c160 tx=0x7fe914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa600c91e6351428 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924001db0 con 0x7fe914005250 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92400ccc0 con 0x7fe914005250 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.470+0000 7fe92a7fc700 1 -- 172.21.15.83:0/3024266999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924003c40 con 0x7fe914005250 2023-11-28T11:12:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 -- 172.21.15.83:0/3024266999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 msgr2=0x7fe91400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe91400ba10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c160 tx=0x7fe914005880 comp rx=0 tx=0).stop 2023-11-28T11:12:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 -- 172.21.15.83:0/3024266999 shutdown_connections 2023-11-28T11:12:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 --2- 172.21.15.83:0/3024266999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe91400ba10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 -- 172.21.15.83:0/3024266999 >> 172.21.15.83:0/3024266999 conn(0x7fe92c0b2b60 msgr2=0x7fe92c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 -- 172.21.15.83:0/3024266999 shutdown_connections 2023-11-28T11:12:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.471+0000 7fe931eff700 1 -- 172.21.15.83:0/3024266999 wait complete. 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.472+0000 7fe931eff700 1 Processor -- start 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.472+0000 7fe931eff700 1 -- start start 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe931eff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe92c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe931eff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe92c1f3680 0x7fe92c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe931eff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe92c150d20 0x7fe92c0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe931eff700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c1f31f0 con 0x7fe914005250 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe931eff700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0ccd30 con 0x7fe92c150d20 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe931eff700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c150ad0 con 0x7fe92c1f3680 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe92b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe92c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe92b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe92c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53278/0 (socket says 172.21.15.83:53278) 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.473+0000 7fe92b7fe700 1 -- 172.21.15.83:0/4115406019 learned_addr learned my addr 172.21.15.83:0/4115406019 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe92b7fe700 1 -- 172.21.15.83:0/4115406019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe92c1f3680 msgr2=0x7fe92c0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe92b7fe700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe92c1f3680 0x7fe92c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe92b7fe700 1 -- 172.21.15.83:0/4115406019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe92c150d20 msgr2=0x7fe92c0b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe92b7fe700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe92c150d20 0x7fe92c0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe92b7fe700 1 -- 172.21.15.83:0/4115406019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0b5520 con 0x7fe914005250 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe92b7fe700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe92c1501b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe920011ca0 tx=0x7fe92000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92000fb30 con 0x7fe914005250 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92000fc90 con 0x7fe914005250 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0b5240 con 0x7fe914005250 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920004020 con 0x7fe914005250 2023-11-28T11:12:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.474+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0b5800 con 0x7fe914005250 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.475+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c1f34d0 con 0x7fe914005250 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.475+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe920024070 con 0x7fe914005250 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.476+0000 7fe913fff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe9180682f0 0x7fe91806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.476+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe9200862a0 con 0x7fe914005250 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.476+0000 7fe92affd700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe9180682f0 0x7fe91806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.476+0000 7fe92affd700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe9180682f0 0x7fe91806a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0b0530 tx=0x7fe91c000ff0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.480+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe920052980 con 0x7fe914005250 2023-11-28T11:12:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.663+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe92c0742b0 con 0x7fe914005250 2023-11-28T11:12:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.663+0000 7fe913fff700 1 -- 172.21.15.83:0/4115406019 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe920022020 con 0x7fe914005250 2023-11-28T11:12:31.665 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:12:31.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe9180682f0 msgr2=0x7fe91806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe9180682f0 0x7fe91806a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0b0530 tx=0x7fe91c000ff0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 msgr2=0x7fe92c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe92c1501b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe920011ca0 tx=0x7fe92000f520 comp rx=0 tx=0).stop 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 shutdown_connections 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe92c150d20 0x7fe92c0b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fe9180682f0 0x7fe91806a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914005250 0x7fe92c1501b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 --2- 172.21.15.83:0/4115406019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe92c1f3680 0x7fe92c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 >> 172.21.15.83:0/4115406019 conn(0x7fe92c0b2b60 msgr2=0x7fe92c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 shutdown_connections 2023-11-28T11:12:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.665+0000 7fe931eff700 1 -- 172.21.15.83:0/4115406019 wait complete. 2023-11-28T11:12:31.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:12:31.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.800+0000 7fb92e780700 1 Processor -- start 2023-11-28T11:12:31.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.800+0000 7fb92e780700 1 -- start start 2023-11-28T11:12:31.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92e780700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb92806dc80 0x7fb92806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92e780700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb92815ca90 0x7fb9280b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92e780700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb9280aed50 0x7fb9280b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92e780700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280b3840 con 0x7fb92815ca90 2023-11-28T11:12:31.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92e780700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280d0a00 con 0x7fb92806dc80 2023-11-28T11:12:31.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92e780700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280ce230 con 0x7fb9280aed50 2023-11-28T11:12:31.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb92815ca90 0x7fb9280b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb92815ca90 0x7fb9280b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53288/0 (socket says 172.21.15.83:53288) 2023-11-28T11:12:31.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 -- 172.21.15.83:0/2569408216 learned_addr learned my addr 172.21.15.83:0/2569408216 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:31.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb927fff700 1 --2- 172.21.15.83:0/2569408216 >> v2:172.21.15.90:3300/0 conn(0x7fb92806dc80 0x7fb92806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb92cd1d700 1 --2- 172.21.15.83:0/2569408216 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb9280aed50 0x7fb9280b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 -- 172.21.15.83:0/2569408216 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb9280aed50 msgr2=0x7fb9280b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 --2- 172.21.15.83:0/2569408216 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb9280aed50 0x7fb9280b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 -- 172.21.15.83:0/2569408216 >> v2:172.21.15.90:3300/0 conn(0x7fb92806dc80 msgr2=0x7fb92806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 --2- 172.21.15.83:0/2569408216 >> v2:172.21.15.90:3300/0 conn(0x7fb92806dc80 0x7fb92806e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 -- 172.21.15.83:0/2569408216 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92806e590 con 0x7fb92815ca90 2023-11-28T11:12:31.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb9277fe700 1 --2- 172.21.15.83:0/2569408216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb9280b1660 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb9280b5c90 tx=0x7fb918009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33ea2d1fd6fdf77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.801+0000 7fb926ffd700 1 -- 172.21.15.83:0/2569408216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800b860 con 0x7fb92815ca90 2023-11-28T11:12:31.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb926ffd700 1 -- 172.21.15.83:0/2569408216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91800b9c0 con 0x7fb92815ca90 2023-11-28T11:12:31.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb926ffd700 1 -- 172.21.15.83:0/2569408216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800bd50 con 0x7fb92815ca90 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 -- 172.21.15.83:0/2569408216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 msgr2=0x7fb9280b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 --2- 172.21.15.83:0/2569408216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb9280b1660 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb9280b5c90 tx=0x7fb918009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 -- 172.21.15.83:0/2569408216 shutdown_connections 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 --2- 172.21.15.83:0/2569408216 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb9280aed50 0x7fb9280b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 --2- 172.21.15.83:0/2569408216 >> v2:172.21.15.90:3300/0 conn(0x7fb92806dc80 0x7fb92806e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 --2- 172.21.15.83:0/2569408216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb9280b1660 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 -- 172.21.15.83:0/2569408216 >> 172.21.15.83:0/2569408216 conn(0x7fb9280b2b30 msgr2=0x7fb9280b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 -- 172.21.15.83:0/2569408216 shutdown_connections 2023-11-28T11:12:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.802+0000 7fb92e780700 1 -- 172.21.15.83:0/2569408216 wait complete. 2023-11-28T11:12:31.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.803+0000 7fb92e780700 1 Processor -- start 2023-11-28T11:12:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.803+0000 7fb92e780700 1 -- start start 2023-11-28T11:12:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92e780700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92806dc80 0x7fb9280c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92e780700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9280aed50 0x7fb92815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92e780700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb92815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92e780700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815d3a0 con 0x7fb92815ca90 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92e780700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815d590 con 0x7fb9280aed50 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92e780700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815d870 con 0x7fb92806dc80 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92cd1d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb92815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92cd1d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb92815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53296/0 (socket says 172.21.15.83:53296) 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb92cd1d700 1 -- 172.21.15.83:0/1362427118 learned_addr learned my addr 172.21.15.83:0/1362427118 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb927fff700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92806dc80 0x7fb9280c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.804+0000 7fb9277fe700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9280aed50 0x7fb92815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92cd1d700 1 -- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92806dc80 msgr2=0x7fb9280c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92cd1d700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92806dc80 0x7fb9280c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92cd1d700 1 -- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9280aed50 msgr2=0x7fb92815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92cd1d700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9280aed50 0x7fb92815ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92cd1d700 1 -- 172.21.15.83:0/1362427118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009550 con 0x7fb92815ca90 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92cd1d700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb92815b910 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb920007e40 tx=0x7fb92000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000ed30 con 0x7fb92815ca90 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920014420 con 0x7fb92815ca90 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280d0160 con 0x7fb92815ca90 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9200147d0 con 0x7fb92815ca90 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.805+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9280cfe80 con 0x7fb92815ca90 2023-11-28T11:12:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.806+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9280d0440 con 0x7fb92815ca90 2023-11-28T11:12:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.806+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb920012050 con 0x7fb92815ca90 2023-11-28T11:12:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.807+0000 7fb924ff9700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9140682f0 0x7fb91406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.807+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb920086480 con 0x7fb92815ca90 2023-11-28T11:12:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.807+0000 7fb927fff700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9140682f0 0x7fb91406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.807+0000 7fb927fff700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9140682f0 0x7fb91406a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9281602b0 tx=0x7fb91c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.811+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb920052a70 con 0x7fb92815ca90 2023-11-28T11:12:31.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.997+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9280742b0 con 0x7fb92815ca90 2023-11-28T11:12:31.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.997+0000 7fb924ff9700 1 -- 172.21.15.83:0/1362427118 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb920056090 con 0x7fb92815ca90 2023-11-28T11:12:31.999 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:12:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9140682f0 msgr2=0x7fb91406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9140682f0 0x7fb91406a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9281602b0 tx=0x7fb91c008040 comp rx=0 tx=0).stop 2023-11-28T11:12:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 msgr2=0x7fb92815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb92815b910 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb920007e40 tx=0x7fb92000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 shutdown_connections 2023-11-28T11:12:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb9140682f0 0x7fb91406a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92815ca90 0x7fb92815b910 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:31.999+0000 7fb92e780700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92806dc80 0x7fb9280c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.000+0000 7fb92e780700 1 --2- 172.21.15.83:0/1362427118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9280aed50 0x7fb92815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.000+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 >> 172.21.15.83:0/1362427118 conn(0x7fb9280b2b30 msgr2=0x7fb9280adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.000+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 shutdown_connections 2023-11-28T11:12:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.000+0000 7fb92e780700 1 -- 172.21.15.83:0/1362427118 wait complete. 2023-11-28T11:12:32.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:12:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.132+0000 7f2be9512700 1 Processor -- start 2023-11-28T11:12:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.132+0000 7f2be9512700 1 -- start start 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.133+0000 7f2be9512700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2be415ed40 0x7f2be40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.133+0000 7f2be9512700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40cf400 con 0x7f2be415a640 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.133+0000 7f2be9512700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be415aa10 con 0x7f2be40cd770 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.133+0000 7f2be9512700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40cbc00 con 0x7f2be415ed40 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.133+0000 7f2be2ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2be415a640 0x7f2be40cb3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51160/0 (socket says 172.21.15.83:51160) 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.133+0000 7f2be2ffd700 1 -- 172.21.15.83:0/149247604 learned_addr learned my addr 172.21.15.83:0/149247604 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203285981 0 0) 0x7f2be40cf400 con 0x7f2be415a640 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be27fc700 1 --2- 172.21.15.83:0/149247604 >> v2:172.21.15.90:3300/0 conn(0x7f2be415ed40 0x7f2be40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd00035b0 con 0x7f2be415a640 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4170543412 0 0) 0x7f2bd00035b0 con 0x7f2be415a640 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 >> v2:172.21.15.90:3300/0 conn(0x7f2be415ed40 msgr2=0x7f2be40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 --2- 172.21.15.83:0/149247604 >> v2:172.21.15.90:3300/0 conn(0x7f2be415ed40 0x7f2be40cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 >> v1:172.21.15.90:6789/0 conn(0x7f2be40cd770 legacy=0x7f2be40b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40cead0 con 0x7f2be415a640 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f2bcc001670 con 0x7f2be415a640 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 >> v1:172.21.15.83:6789/0 conn(0x7f2be415a640 legacy=0x7f2be40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd0003a00 0x7f2bd0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0004850 0x7f2bd0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bd0005200 0x7f2bd000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40cbc00 con 0x7f2bd0003a00 2023-11-28T11:12:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000c080 con 0x7f2bd0004850 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000c240 con 0x7f2bd0005200 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd0003a00 0x7f2bd0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be27fc700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0004850 0x7f2bd0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be37fe700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bd0005200 0x7f2bd000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be2ffd700 1 -- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bd0005200 msgr2=0x7f2bd000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bd0005200 0x7f2bd000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be2ffd700 1 -- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0004850 msgr2=0x7f2bd0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0004850 0x7f2bd0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be2ffd700 1 -- 172.21.15.83:0/149247604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00035b0 con 0x7f2bd0003a00 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.134+0000 7f2be37fe700 1 -- 172.21.15.83:0/149247604 reap_dead start 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd0003a00 0x7f2bd0003e10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c240 tx=0x7f2bd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b37b88761aeff4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc007c10 con 0x7f2bd0003a00 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc00cde0 con 0x7f2bd0003a00 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be1ffb700 1 -- 172.21.15.83:0/149247604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc005ba0 con 0x7f2bd0003a00 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 -- 172.21.15.83:0/149247604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd0003a00 msgr2=0x7f2bd0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd0003a00 0x7f2bd0003e10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c240 tx=0x7f2bd0005880 comp rx=0 tx=0).stop 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 -- 172.21.15.83:0/149247604 shutdown_connections 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 --2- 172.21.15.83:0/149247604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd0003a00 0x7f2bd0003e10 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 -- 172.21.15.83:0/149247604 >> 172.21.15.83:0/149247604 conn(0x7f2be406f300 msgr2=0x7f2be406f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 -- 172.21.15.83:0/149247604 shutdown_connections 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.135+0000 7f2be9512700 1 -- 172.21.15.83:0/149247604 wait complete. 2023-11-28T11:12:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 Processor -- start 2023-11-28T11:12:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 -- start start 2023-11-28T11:12:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0003a00 0x7f2be41f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 0x7f2be4158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2be4158bc0 0x7f2be41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4159030 con 0x7f2be40b67f0 2023-11-28T11:12:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41f2cb0 con 0x7f2bd0003a00 2023-11-28T11:12:32.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be9512700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41e8630 con 0x7f2be4158bc0 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 0x7f2be4158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.137+0000 7f2be27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 0x7f2be4158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53316/0 (socket says 172.21.15.83:53316) 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 -- 172.21.15.83:0/181220866 learned_addr learned my addr 172.21.15.83:0/181220866 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0003a00 0x7f2be41f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be37fe700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2be4158bc0 0x7f2be41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 -- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2be4158bc0 msgr2=0x7f2be41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2be4158bc0 0x7f2be41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 -- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0003a00 msgr2=0x7f2be41f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0003a00 0x7f2be41f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 -- 172.21.15.83:0/181220866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40cf090 con 0x7f2be40b67f0 2023-11-28T11:12:32.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0003a00 0x7f2be41f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be27fc700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 0x7f2be4158680 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4011aa0 tx=0x7f2bd4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400f980 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd400fb00 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be40cedb0 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400feb0 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.138+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be40cdbf0 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.139+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be40742b0 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.139+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2bd4014440 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.140+0000 7f2bdb7fe700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2bc40682f0 0x7f2bc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.140+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2bd4086670 con 0x7f2be40b67f0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.140+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2bc40682f0 0x7f2bc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.140+0000 7f2be2ffd700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2bc40682f0 0x7f2bc406a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000c00 tx=0x7f2bcc007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.144+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2bd4052db0 con 0x7f2be40b67f0 2023-11-28T11:12:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.340+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2be41565b0 con 0x7f2be40b67f0 2023-11-28T11:12:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.340+0000 7f2bdb7fe700 1 -- 172.21.15.83:0/181220866 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2bd40563d0 con 0x7f2be40b67f0 2023-11-28T11:12:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2bc40682f0 msgr2=0x7f2bc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2bc40682f0 0x7f2bc406a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000c00 tx=0x7f2bcc007000 comp rx=0 tx=0).stop 2023-11-28T11:12:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 msgr2=0x7f2be4158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 0x7f2be4158680 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4011aa0 tx=0x7f2bd4011e60 comp rx=0 tx=0).stop 2023-11-28T11:12:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 shutdown_connections 2023-11-28T11:12:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2be4158bc0 0x7f2be41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd0003a00 0x7f2be41f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2bc40682f0 0x7f2bc406a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 --2- 172.21.15.83:0/181220866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b67f0 0x7f2be4158680 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 >> 172.21.15.83:0/181220866 conn(0x7f2be406f300 msgr2=0x7f2be415aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 shutdown_connections 2023-11-28T11:12:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.342+0000 7f2be9512700 1 -- 172.21.15.83:0/181220866 wait complete. 2023-11-28T11:12:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-28T11:12:32.356 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:12:32.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:32.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:32.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:12:32.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:32.418 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:12:32.452 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:12:32.452 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:32.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:32.471 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:12:32.471 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:32.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:12:32.510 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:12:32.510 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:32.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:32.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:32.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.669+0000 7f97c79b2700 1 Processor -- start 2023-11-28T11:12:32.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.669+0000 7f97c79b2700 1 -- start start 2023-11-28T11:12:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.670+0000 7f97c79b2700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f97c015d0e0 0x7f97c00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.670+0000 7f97c79b2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 0x7f97c00cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.670+0000 7f97c79b2700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00cbfb0 con 0x7f97c015cd10 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.670+0000 7f97c79b2700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c006cee0 con 0x7f97c015d0e0 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.670+0000 7f97c79b2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00cb710 con 0x7f97c00b1fb0 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.670+0000 7f97c5f4f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 0x7f97c00cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c4f4d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f97c015d0e0 0x7f97c00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 0x7f97c00cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:37528/0 (socket says 172.21.15.83:37528) 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 -- 172.21.15.83:0/4076567888 learned_addr learned my addr 172.21.15.83:0/4076567888 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c574e700 1 --1- 172.21.15.83:0/4076567888 >> v1:172.21.15.90:6789/0 conn(0x7f97c015cd10 0x7f97c00b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:49528/0 (socket says 172.21.15.83:49528) 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97b7fff700 1 -- 172.21.15.83:0/4076567888 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330168454 0 0) 0x7f97c00cbfb0 con 0x7f97c015cd10 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97b7fff700 1 -- 172.21.15.83:0/4076567888 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97ac0035b0 con 0x7f97c015cd10 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 -- 172.21.15.83:0/4076567888 >> v2:172.21.15.90:3300/0 conn(0x7f97c015d0e0 msgr2=0x7f97c00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 --2- 172.21.15.83:0/4076567888 >> v2:172.21.15.90:3300/0 conn(0x7f97c015d0e0 0x7f97c00ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 -- 172.21.15.83:0/4076567888 >> v1:172.21.15.90:6789/0 conn(0x7f97c015cd10 legacy=0x7f97c00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 -- 172.21.15.83:0/4076567888 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00cb4e0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c4f4d700 1 --2- 172.21.15.83:0/4076567888 >> v2:172.21.15.90:3300/0 conn(0x7f97c015d0e0 0x7f97c00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97c5f4f700 1 --2- 172.21.15.83:0/4076567888 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 0x7f97c00cafa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009e50 tx=0x7f97b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f759c09c6d89ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97b7fff700 1 -- 172.21.15.83:0/4076567888 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800e070 con 0x7f97c00b1fb0 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97b7fff700 1 -- 172.21.15.83:0/4076567888 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b800b910 con 0x7f97c00b1fb0 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.671+0000 7f97b7fff700 1 -- 172.21.15.83:0/4076567888 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800bc80 con 0x7f97c00b1fb0 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 -- 172.21.15.83:0/4076567888 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 msgr2=0x7f97c00cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 --2- 172.21.15.83:0/4076567888 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 0x7f97c00cafa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009e50 tx=0x7f97b800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 -- 172.21.15.83:0/4076567888 shutdown_connections 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 --2- 172.21.15.83:0/4076567888 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c00b1fb0 0x7f97c00cafa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 --2- 172.21.15.83:0/4076567888 >> v2:172.21.15.90:3300/0 conn(0x7f97c015d0e0 0x7f97c00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 -- 172.21.15.83:0/4076567888 >> 172.21.15.83:0/4076567888 conn(0x7f97c006e9a0 msgr2=0x7f97c00cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 -- 172.21.15.83:0/4076567888 shutdown_connections 2023-11-28T11:12:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.672+0000 7f97c79b2700 1 -- 172.21.15.83:0/4076567888 wait complete. 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.673+0000 7f97c79b2700 1 Processor -- start 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.673+0000 7f97c79b2700 1 -- start start 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 0x7f97c00b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c015cd10 0x7f97c015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97c015d0e0 0x7f97c00c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015ed40 con 0x7f97c00b1fb0 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00c8420 con 0x7f97c015d0e0 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00c8700 con 0x7f97c015cd10 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 0x7f97c00b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 0x7f97c00b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53328/0 (socket says 172.21.15.83:53328) 2023-11-28T11:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 -- 172.21.15.83:0/2422546877 learned_addr learned my addr 172.21.15.83:0/2422546877 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c5f4f700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97c015d0e0 0x7f97c00c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c4f4d700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c015cd10 0x7f97c015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 -- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c015cd10 msgr2=0x7f97c015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c015cd10 0x7f97c015e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 -- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97c015d0e0 msgr2=0x7f97c00c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97c015d0e0 0x7f97c00c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 -- 172.21.15.83:0/2422546877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0094a0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c574e700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 0x7f97c00b2c50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c500 tx=0x7f97b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0005cb0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b0005e10 con 0x7f97c00b1fb0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.674+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b8009b30 con 0x7f97c00b1fb0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.675+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0007310 con 0x7f97c00b1fb0 2023-11-28T11:12:32.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.675+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c00cbcd0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.676+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c00cb9f0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.676+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97b0019030 con 0x7f97c00b1fb0 2023-11-28T11:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.676+0000 7f97b5ffb700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97a40682f0 0x7f97a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.677+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f97b00850b0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.677+0000 7f97c4f4d700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97a40682f0 0x7f97a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.677+0000 7f97c4f4d700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97a40682f0 0x7f97a406a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f97c015baf0 tx=0x7f97bc009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.681+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97b0051850 con 0x7f97c00b1fb0 2023-11-28T11:12:32.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.906+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97c006b9b0 con 0x7f97c00b1fb0 2023-11-28T11:12:32.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.907+0000 7f97b5ffb700 1 -- 172.21.15.83:0/2422546877 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f97b0054e70 con 0x7f97c00b1fb0 2023-11-28T11:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:40.850456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97a40682f0 msgr2=0x7f97a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97a40682f0 0x7f97a406a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f97c015baf0 tx=0x7f97bc009210 comp rx=0 tx=0).stop 2023-11-28T11:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 msgr2=0x7f97c00b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 0x7f97c00b2c50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c500 tx=0x7f97b000c810 comp rx=0 tx=0).stop 2023-11-28T11:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 shutdown_connections 2023-11-28T11:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97c015d0e0 0x7f97c00c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f97a40682f0 0x7f97a406a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c00b1fb0 0x7f97c00b2c50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 --2- 172.21.15.83:0/2422546877 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f97c015cd10 0x7f97c015e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 >> 172.21.15.83:0/2422546877 conn(0x7f97c006e9a0 msgr2=0x7f97c00b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 shutdown_connections 2023-11-28T11:12:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:32.909+0000 7f97c79b2700 1 -- 172.21.15.83:0/2422546877 wait complete. 2023-11-28T11:12:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2023-11-28T11:12:32.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.042+0000 7f1b8e7bd700 1 Processor -- start 2023-11-28T11:12:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.042+0000 7f1b8e7bd700 1 -- start start 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.043+0000 7f1b8e7bd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b88159830 0x7f1b880cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.043+0000 7f1b8e7bd700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 0x7f1b880b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.043+0000 7f1b8e7bd700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8806ca60 con 0x7f1b8806c5f0 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.043+0000 7f1b8e7bd700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8815d480 con 0x7f1b880b18a0 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.043+0000 7f1b8e7bd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880b2260 con 0x7f1b88159830 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b8cd5a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 0x7f1b880b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b87fff700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b88159830 0x7f1b880cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b8cd5a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 0x7f1b880b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.83:40334/0 (socket says 172.21.15.83:40334) 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b8cd5a700 1 -- 172.21.15.83:0/3282891871 learned_addr learned my addr 172.21.15.83:0/3282891871 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b87fff700 1 -- 172.21.15.83:0/3282891871 >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 msgr2=0x7f1b880b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b87fff700 1 --2- 172.21.15.83:0/3282891871 >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 0x7f1b880b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b87fff700 1 -- 172.21.15.83:0/3282891871 >> v1:172.21.15.90:6789/0 conn(0x7f1b8806c5f0 legacy=0x7f1b8815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b87fff700 1 -- 172.21.15.83:0/3282891871 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880cf790 con 0x7f1b88159830 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b8cd5a700 1 --2- 172.21.15.83:0/3282891871 >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 0x7f1b880b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b87fff700 1 --2- 172.21.15.83:0/3282891871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b88159830 0x7f1b880cf0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009840 tx=0x7f1b7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c6aca5a3c94a72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b86ffd700 1 -- 172.21.15.83:0/3282891871 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011830 con 0x7f1b88159830 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b86ffd700 1 -- 172.21.15.83:0/3282891871 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c011990 con 0x7f1b88159830 2023-11-28T11:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.044+0000 7f1b86ffd700 1 -- 172.21.15.83:0/3282891871 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011d20 con 0x7f1b88159830 2023-11-28T11:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/3282891871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b88159830 msgr2=0x7f1b880cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/3282891871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b88159830 0x7f1b880cf0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009840 tx=0x7f1b7c009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/3282891871 shutdown_connections 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/3282891871 >> v2:172.21.15.90:3300/0 conn(0x7f1b880b18a0 0x7f1b880b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/3282891871 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b88159830 0x7f1b880cf0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/3282891871 >> 172.21.15.83:0/3282891871 conn(0x7f1b8806e9a0 msgr2=0x7f1b880cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/3282891871 shutdown_connections 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.045+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/3282891871 wait complete. 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.046+0000 7f1b8e7bd700 1 Processor -- start 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.047+0000 7f1b8e7bd700 1 -- start start 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.047+0000 7f1b8e7bd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8806c5f0 0x7f1b88153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.047+0000 7f1b8e7bd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 0x7f1b8815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.047+0000 7f1b8e7bd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88159830 0x7f1b88155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b8e7bd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88153a70 con 0x7f1b880b18a0 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b8e7bd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88155980 con 0x7f1b88159830 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b8e7bd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b881e2d40 con 0x7f1b8806c5f0 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 0x7f1b8815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 0x7f1b8815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53344/0 (socket says 172.21.15.83:53344) 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 -- 172.21.15.83:0/2925639893 learned_addr learned my addr 172.21.15.83:0/2925639893 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b8cd5a700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88159830 0x7f1b88155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b87fff700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8806c5f0 0x7f1b88153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 -- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8806c5f0 msgr2=0x7f1b88153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8806c5f0 0x7f1b88153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 -- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88159830 msgr2=0x7f1b88155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88159830 0x7f1b88155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 -- 172.21.15.83:0/2925639893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80009b30 con 0x7f1b880b18a0 2023-11-28T11:12:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b877fe700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 0x7f1b8815bc00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c450 tx=0x7f1b7800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b780052d0 con 0x7f1b880b18a0 2023-11-28T11:12:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78005430 con 0x7f1b880b18a0 2023-11-28T11:12:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c0094a0 con 0x7f1b880b18a0 2023-11-28T11:12:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b780038c0 con 0x7f1b880b18a0 2023-11-28T11:12:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.048+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8815d690 con 0x7f1b880b18a0 2023-11-28T11:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.049+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b880cfa70 con 0x7f1b880b18a0 2023-11-28T11:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.050+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b78019030 con 0x7f1b880b18a0 2023-11-28T11:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.050+0000 7f1b84ff9700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b740682f0 0x7f1b7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.050+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1b78085040 con 0x7f1b880b18a0 2023-11-28T11:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.050+0000 7f1b87fff700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b740682f0 0x7f1b7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.050+0000 7f1b87fff700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b740682f0 0x7f1b7406a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c000c00 tx=0x7f1b7c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.054+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b78051690 con 0x7f1b880b18a0 2023-11-28T11:12:33.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.237+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b8815d970 con 0x7f1b880b18a0 2023-11-28T11:12:33.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.238+0000 7f1b84ff9700 1 -- 172.21.15.83:0/2925639893 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18175 (secure 0 0 0) 0x7f1b78017020 con 0x7f1b880b18a0 2023-11-28T11:12:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:33.240 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:12:25.948014+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b740682f0 msgr2=0x7f1b7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b740682f0 0x7f1b7406a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c000c00 tx=0x7f1b7c008030 comp rx=0 tx=0).stop 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 msgr2=0x7f1b8815bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 0x7f1b8815bc00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c450 tx=0x7f1b7800c760 comp rx=0 tx=0).stop 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 shutdown_connections 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88159830 0x7f1b88155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b8806c5f0 0x7f1b88153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b740682f0 0x7f1b7406a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 --2- 172.21.15.83:0/2925639893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b880b18a0 0x7f1b8815bc00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.240+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 >> 172.21.15.83:0/2925639893 conn(0x7f1b8806e9a0 msgr2=0x7f1b880cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:33.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.241+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 shutdown_connections 2023-11-28T11:12:33.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.241+0000 7f1b8e7bd700 1 -- 172.21.15.83:0/2925639893 wait complete. 2023-11-28T11:12:33.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:29.338657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:29.610310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:11:22.194488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:11:23.354896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:12:33.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:12:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.376+0000 7fd87db36700 1 Processor -- start 2023-11-28T11:12:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.376+0000 7fd87db36700 1 -- start start 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd87db36700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8780b0240 0x7fd8780d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd87db36700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd8780cdc50 0x7fd8781608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd87db36700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87806f9a0 con 0x7fd8780cd880 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd87db36700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87806fc80 con 0x7fd8780b0240 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd87db36700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87806ff60 con 0x7fd8780cdc50 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8780b0240 0x7fd8780d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8780b0240 0x7fd8780d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53352/0 (socket says 172.21.15.83:53352) 2023-11-28T11:12:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 -- 172.21.15.83:0/1547431681 learned_addr learned my addr 172.21.15.83:0/1547431681 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd877fff700 1 --2- 172.21.15.83:0/1547431681 >> v2:172.21.15.90:3300/0 conn(0x7fd8780cdc50 0x7fd8781608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 -- 172.21.15.83:0/1547431681 >> v2:172.21.15.90:3300/0 conn(0x7fd8780cdc50 msgr2=0x7fd8781608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 --2- 172.21.15.83:0/1547431681 >> v2:172.21.15.90:3300/0 conn(0x7fd8780cdc50 0x7fd8781608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 -- 172.21.15.83:0/1547431681 >> v1:172.21.15.90:6789/0 conn(0x7fd8780cd880 legacy=0x7fd87806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.377+0000 7fd876ffd700 1 -- 172.21.15.83:0/1547431681 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8781603b0 con 0x7fd8780b0240 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd876ffd700 1 --2- 172.21.15.83:0/1547431681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780b0240 0x7fd8780d0710 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098f0 tx=0x7fd868009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3785502f44ce7be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd8767fc700 1 -- 172.21.15.83:0/1547431681 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86800b860 con 0x7fd8780b0240 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd8767fc700 1 -- 172.21.15.83:0/1547431681 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86800b9c0 con 0x7fd8780b0240 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd8767fc700 1 -- 172.21.15.83:0/1547431681 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86800bd50 con 0x7fd8780b0240 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 -- 172.21.15.83:0/1547431681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780b0240 msgr2=0x7fd8780d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 --2- 172.21.15.83:0/1547431681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780b0240 0x7fd8780d0710 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098f0 tx=0x7fd868009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 -- 172.21.15.83:0/1547431681 shutdown_connections 2023-11-28T11:12:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 --2- 172.21.15.83:0/1547431681 >> v2:172.21.15.90:3300/0 conn(0x7fd8780cdc50 0x7fd8781608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 --2- 172.21.15.83:0/1547431681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780b0240 0x7fd8780d0710 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 -- 172.21.15.83:0/1547431681 >> 172.21.15.83:0/1547431681 conn(0x7fd87806de20 msgr2=0x7fd87806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 -- 172.21.15.83:0/1547431681 shutdown_connections 2023-11-28T11:12:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.378+0000 7fd87db36700 1 -- 172.21.15.83:0/1547431681 wait complete. 2023-11-28T11:12:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.379+0000 7fd87db36700 1 Processor -- start 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 -- start start 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8780b0240 0x7fd8781f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8780cd880 0x7fd878155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 0x7fd87815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781f6a40 con 0x7fd8780cdc50 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87815f900 con 0x7fd8780b0240 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd87db36700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781f61a0 con 0x7fd8780cd880 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.380+0000 7fd877fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 0x7fd87815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 0x7fd87815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53360/0 (socket says 172.21.15.83:53360) 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 -- 172.21.15.83:0/2975545256 learned_addr learned my addr 172.21.15.83:0/2975545256 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd8777fe700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8780b0240 0x7fd8781f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd876ffd700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8780cd880 0x7fd878155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 -- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8780cd880 msgr2=0x7fd878155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8780cd880 0x7fd878155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 -- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8780b0240 msgr2=0x7fd8781f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8780b0240 0x7fd8781f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 -- 172.21.15.83:0/2975545256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009550 con 0x7fd8780cdc50 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd8777fe700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8780b0240 0x7fd8781f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd877fff700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 0x7fd87815f3c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd870007e60 tx=0x7fd87000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87000ede0 con 0x7fd8780cdc50 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd870014420 con 0x7fd8780cdc50 2023-11-28T11:12:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87806c5e0 con 0x7fd8780cdc50 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700147d0 con 0x7fd8780cdc50 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.381+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87806d670 con 0x7fd8780cdc50 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.382+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd878160df0 con 0x7fd8780cdc50 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.382+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd870012050 con 0x7fd8780cdc50 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.383+0000 7fd86ffff700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd8580682f0 0x7fd85806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.383+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd870086440 con 0x7fd8780cdc50 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.383+0000 7fd8777fe700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd8580682f0 0x7fd85806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.383+0000 7fd8777fe700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd8580682f0 0x7fd85806a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd860000f60 tx=0x7fd860009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.387+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd870052b20 con 0x7fd8780cdc50 2023-11-28T11:12:33.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.568+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd8780742b0 con 0x7fd8780cdc50 2023-11-28T11:12:33.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.569+0000 7fd86ffff700 1 -- 172.21.15.83:0/2975545256 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd870056140 con 0x7fd8780cdc50 2023-11-28T11:12:33.570 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2023-11-28T11:12:33.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.570+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd8580682f0 msgr2=0x7fd85806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd8580682f0 0x7fd85806a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd860000f60 tx=0x7fd860009040 comp rx=0 tx=0).stop 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 msgr2=0x7fd87815f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 0x7fd87815f3c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd870007e60 tx=0x7fd87000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 shutdown_connections 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fd8580682f0 0x7fd85806a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8780cdc50 0x7fd87815f3c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8780b0240 0x7fd8781f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 --2- 172.21.15.83:0/2975545256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8780cd880 0x7fd878155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 >> 172.21.15.83:0/2975545256 conn(0x7fd87806de20 msgr2=0x7fd8780b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 shutdown_connections 2023-11-28T11:12:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.571+0000 7fd87db36700 1 -- 172.21.15.83:0/2975545256 wait complete. 2023-11-28T11:12:33.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:12:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.706+0000 7fbdbbe69700 1 Processor -- start 2023-11-28T11:12:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.706+0000 7fbdbbe69700 1 -- start start 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdbbe69700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdb40cc390 0x7fbdb415e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdbbe69700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc760 0x7fbdb40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdbbe69700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbdb415b150 0x7fbdb415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdbbe69700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb415aa40 con 0x7fbdb40cc390 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdbbe69700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb40cbb60 con 0x7fbdb415b150 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdbbe69700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb415acc0 con 0x7fbdb40cc760 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdb40cc390 0x7fbdb415e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdb40cc390 0x7fbdb415e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53364/0 (socket says 172.21.15.83:53364) 2023-11-28T11:12:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 -- 172.21.15.83:0/4142399773 learned_addr learned my addr 172.21.15.83:0/4142399773 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdba406700 1 --2- 172.21.15.83:0/4142399773 >> v2:172.21.15.90:3300/0 conn(0x7fbdb415b150 0x7fbdb415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9404700 1 --2- 172.21.15.83:0/4142399773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc760 0x7fbdb40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 -- 172.21.15.83:0/4142399773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc760 msgr2=0x7fbdb40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 --2- 172.21.15.83:0/4142399773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc760 0x7fbdb40cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 -- 172.21.15.83:0/4142399773 >> v2:172.21.15.90:3300/0 conn(0x7fbdb415b150 msgr2=0x7fbdb415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 --2- 172.21.15.83:0/4142399773 >> v2:172.21.15.90:3300/0 conn(0x7fbdb415b150 0x7fbdb415b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 -- 172.21.15.83:0/4142399773 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb415e730 con 0x7fbdb40cc390 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.707+0000 7fbdb9c05700 1 --2- 172.21.15.83:0/4142399773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc390 0x7fbdb415e030 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40c9940 tx=0x7fbdb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f57cb8e3ca1532 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdb8c03700 1 -- 172.21.15.83:0/4142399773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0011830 con 0x7fbdb40cc390 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdb8c03700 1 -- 172.21.15.83:0/4142399773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0011990 con 0x7fbdb40cc390 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdb8c03700 1 -- 172.21.15.83:0/4142399773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0011d20 con 0x7fbdb40cc390 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 -- 172.21.15.83:0/4142399773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc390 msgr2=0x7fbdb415e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/4142399773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc390 0x7fbdb415e030 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40c9940 tx=0x7fbdb0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 -- 172.21.15.83:0/4142399773 shutdown_connections 2023-11-28T11:12:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/4142399773 >> v2:172.21.15.90:3300/0 conn(0x7fbdb415b150 0x7fbdb415b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/4142399773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc390 0x7fbdb415e030 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/4142399773 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc760 0x7fbdb40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 -- 172.21.15.83:0/4142399773 >> 172.21.15.83:0/4142399773 conn(0x7fbdb40b1090 msgr2=0x7fbdb40b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 -- 172.21.15.83:0/4142399773 shutdown_connections 2023-11-28T11:12:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.708+0000 7fbdbbe69700 1 -- 172.21.15.83:0/4142399773 wait complete. 2023-11-28T11:12:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.709+0000 7fbdbbe69700 1 Processor -- start 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.709+0000 7fbdbbe69700 1 -- start start 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdbbe69700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc390 0x7fbdb40b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdbbe69700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 0x7fbdb41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdbbe69700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb415b150 0x7fbdb4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdbbe69700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4150dc0 con 0x7fbdb40cc760 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdbbe69700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40b5080 con 0x7fbdb415b150 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdbbe69700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40b22e0 con 0x7fbdb40cc390 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdb9404700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 0x7fbdb41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdb9404700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 0x7fbdb41522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53372/0 (socket says 172.21.15.83:53372) 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdb9404700 1 -- 172.21.15.83:0/1823343006 learned_addr learned my addr 172.21.15.83:0/1823343006 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdb9c05700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc390 0x7fbdb40b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.710+0000 7fbdba406700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb415b150 0x7fbdb4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdb9404700 1 -- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc390 msgr2=0x7fbdb40b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdb9404700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc390 0x7fbdb40b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdb9404700 1 -- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb415b150 msgr2=0x7fbdb4150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdb9404700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb415b150 0x7fbdb4150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdb9404700 1 -- 172.21.15.83:0/1823343006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0009550 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdb9404700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 0x7fbdb41522a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c7f0 tx=0x7fbda400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4004220 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda4004380 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb415ecf0 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda40036a0 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.711+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb415ea10 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.712+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb4150f30 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.712+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbda4020070 con 0x7fbdb40cc760 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.713+0000 7fbdaa7fc700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd98068340 0x7fbd9806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.713+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbda4085670 con 0x7fbdb40cc760 2023-11-28T11:12:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.713+0000 7fbdb9c05700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd98068340 0x7fbd9806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.713+0000 7fbdb9c05700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd98068340 0x7fbd9806a7f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009c10 tx=0x7fbdb0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.717+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbda4051df0 con 0x7fbdb40cc760 2023-11-28T11:12:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.898+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdb40742b0 con 0x7fbdb40cc760 2023-11-28T11:12:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.899+0000 7fbdaa7fc700 1 -- 172.21.15.83:0/1823343006 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbda4055410 con 0x7fbdb40cc760 2023-11-28T11:12:33.900 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-28T11:12:33.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd98068340 msgr2=0x7fbd9806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd98068340 0x7fbd9806a7f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009c10 tx=0x7fbdb0008030 comp rx=0 tx=0).stop 2023-11-28T11:12:33.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 msgr2=0x7fbdb41522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 0x7fbdb41522a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c7f0 tx=0x7fbda400cb00 comp rx=0 tx=0).stop 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 shutdown_connections 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb415b150 0x7fbdb4150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbdb40cc390 0x7fbdb40b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fbd98068340 0x7fbd9806a7f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 --2- 172.21.15.83:0/1823343006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb40cc760 0x7fbdb41522a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 >> 172.21.15.83:0/1823343006 conn(0x7fbdb40b1090 msgr2=0x7fbdb415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 shutdown_connections 2023-11-28T11:12:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:33.901+0000 7fbdbbe69700 1 -- 172.21.15.83:0/1823343006 wait complete. 2023-11-28T11:12:33.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:12:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.036+0000 7f8193aac700 1 Processor -- start 2023-11-28T11:12:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.036+0000 7f8193aac700 1 -- start start 2023-11-28T11:12:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.037+0000 7f8193aac700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 0x7f818c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.037+0000 7f8193aac700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c06d4a0 con 0x7f818c06e370 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.037+0000 7f8193aac700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0b5ae0 con 0x7f818c15c890 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.037+0000 7f8193aac700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c06c400 con 0x7f818c0aee60 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.037+0000 7f8191047700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f818c06e370 0x7f818c06b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51240/0 (socket says 172.21.15.83:51240) 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.037+0000 7f8191047700 1 -- 172.21.15.83:0/917125575 learned_addr learned my addr 172.21.15.83:0/917125575 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8192049700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 0x7f818c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966641165 0 0) 0x7f818c06d4a0 con 0x7f818c06e370 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81780035b0 con 0x7f818c06e370 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871620311 0 0) 0x7f818c0b5ae0 con 0x7f818c15c890 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f818c06d4a0 con 0x7f818c15c890 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2130296925 0 0) 0x7f81780035b0 con 0x7f818c06e370 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 msgr2=0x7f818c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 0x7f818c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 >> v1:172.21.15.90:6789/0 conn(0x7f818c15c890 legacy=0x7f818c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0b5240 con 0x7f818c06e370 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f817c001670 con 0x7f818c06e370 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 >> v1:172.21.15.83:6789/0 conn(0x7f818c06e370 legacy=0x7f818c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 0x7f8178003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f8178004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8178004d90 0x7f8178005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c06c400 con 0x7f8178004120 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817800bba0 con 0x7f8178004d90 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817800bd30 con 0x7f818c0aee60 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191047700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f8178004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191848700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 0x7f8178003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8192049700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8178004d90 0x7f8178005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191047700 1 -- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 msgr2=0x7f8178003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191047700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0aee60 0x7f8178003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191047700 1 -- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8178004d90 msgr2=0x7f8178005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191047700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8178004d90 0x7f8178005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.038+0000 7f8191047700 1 -- 172.21.15.83:0/917125575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81780035b0 con 0x7f8178004120 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8191047700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f8178004850 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f818c06e760 tx=0x7f8178005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f1965543c54674 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00cce0 con 0x7f8178004120 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00ce60 con 0x7f8178004120 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8190846700 1 -- 172.21.15.83:0/917125575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c005ba0 con 0x7f8178004120 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8193aac700 1 -- 172.21.15.83:0/917125575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 msgr2=0x7f8178004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8193aac700 1 --2- 172.21.15.83:0/917125575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f8178004850 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f818c06e760 tx=0x7f8178005880 comp rx=0 tx=0).stop 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8192049700 1 -- 172.21.15.83:0/917125575 reap_dead start 2023-11-28T11:12:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8193aac700 1 -- 172.21.15.83:0/917125575 shutdown_connections 2023-11-28T11:12:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8193aac700 1 -- 172.21.15.83:0/917125575 >> 172.21.15.83:0/917125575 conn(0x7f818c0b2b60 msgr2=0x7f818c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8193aac700 1 -- 172.21.15.83:0/917125575 shutdown_connections 2023-11-28T11:12:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.039+0000 7f8193aac700 1 -- 172.21.15.83:0/917125575 wait complete. 2023-11-28T11:12:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 Processor -- start 2023-11-28T11:12:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 -- start start 2023-11-28T11:12:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f818c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f818c1590b0 0x7f818c0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0c6270 0x7f818c0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0c5760 con 0x7f8178004120 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0c4ae0 con 0x7f818c1590b0 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8193aac700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0c4dc0 con 0x7f818c0c6270 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8191848700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f818c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8191848700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f818c15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53392/0 (socket says 172.21.15.83:53392) 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.041+0000 7f8191848700 1 -- 172.21.15.83:0/2271067420 learned_addr learned my addr 172.21.15.83:0/2271067420 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191047700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f818c1590b0 0x7f818c0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8192049700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0c6270 0x7f818c0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191848700 1 -- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0c6270 msgr2=0x7f818c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191848700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0c6270 0x7f818c0aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191848700 1 -- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f818c1590b0 msgr2=0x7f818c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191848700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f818c1590b0 0x7f818c0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191848700 1 -- 172.21.15.83:0/2271067420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0b5800 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8191848700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f818c15d9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f818800f7a0 tx=0x7f818800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81880052e0 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188005440 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c0b5520 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188003910 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.042+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c06ac30 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.043+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c0af670 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.043+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8188003a70 con 0x7f8178004120 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.044+0000 7f81827fc700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81700682f0 0x7f817006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.044+0000 7f8191047700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81700682f0 0x7f817006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.044+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f8188085eb0 con 0x7f8178004120 2023-11-28T11:12:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.044+0000 7f8191047700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81700682f0 0x7f817006a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c00 tx=0x7f817c007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.048+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8188052590 con 0x7f8178004120 2023-11-28T11:12:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.231+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f818c0b3820 con 0x7f8178004120 2023-11-28T11:12:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.232+0000 7f81827fc700 1 -- 172.21.15.83:0/2271067420 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8188055bb0 con 0x7f8178004120 2023-11-28T11:12:34.233 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-28T11:12:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.233+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81700682f0 msgr2=0x7f817006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.233+0000 7f8193aac700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81700682f0 0x7f817006a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c00 tx=0x7f817c007f90 comp rx=0 tx=0).stop 2023-11-28T11:12:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 msgr2=0x7f818c15d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f818c15d9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f818800f7a0 tx=0x7f818800fb60 comp rx=0 tx=0).stop 2023-11-28T11:12:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 shutdown_connections 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f818c0c6270 0x7f818c0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81700682f0 0x7f817006a7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8178004120 0x7f818c15d9c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 --2- 172.21.15.83:0/2271067420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f818c1590b0 0x7f818c0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 >> 172.21.15.83:0/2271067420 conn(0x7f818c0b2b60 msgr2=0x7f818c0cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 shutdown_connections 2023-11-28T11:12:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.234+0000 7f8193aac700 1 -- 172.21.15.83:0/2271067420 wait complete. 2023-11-28T11:12:34.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:12:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.368+0000 7f9d11f16700 1 Processor -- start 2023-11-28T11:12:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.368+0000 7f9d11f16700 1 -- start start 2023-11-28T11:12:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d11f16700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 0x7f9d0c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d11f16700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9d0c160500 0x7f9d0c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d11f16700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0cc4b0 con 0x7f9d0c06e6a0 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d11f16700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c06f420 con 0x7f9d0c160500 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d11f16700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c160ec0 con 0x7f9d0c06ea70 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d0b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9d0c06e6a0 0x7f9d0c0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51258/0 (socket says 172.21.15.83:51258) 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d0b7fe700 1 -- 172.21.15.83:0/14225844 learned_addr learned my addr 172.21.15.83:0/14225844 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d0affd700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 0x7f9d0c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d0bfff700 1 --2- 172.21.15.83:0/14225844 >> v2:172.21.15.90:3300/0 conn(0x7f9d0c160500 0x7f9d0c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57883644 0 0) 0x7f9d0c0cc4b0 con 0x7f9d0c06e6a0 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.369+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf80035b0 con 0x7f9d0c06e6a0 2023-11-28T11:12:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 663233083 0 0) 0x7f9cf80035b0 con 0x7f9d0c06e6a0 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 msgr2=0x7f9d0c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 0x7f9d0c06eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 >> v2:172.21.15.90:3300/0 conn(0x7f9d0c160500 msgr2=0x7f9d0c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 --2- 172.21.15.83:0/14225844 >> v2:172.21.15.90:3300/0 conn(0x7f9d0c160500 0x7f9d0c160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0cc790 con 0x7f9d0c06e6a0 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f9cf40017f0 con 0x7f9d0c06e6a0 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 >> v1:172.21.15.83:6789/0 conn(0x7f9d0c06e6a0 legacy=0x7f9d0c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 0x7f9cf8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf8003f80 0x7f9cf8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9cf8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c06f420 con 0x7f9cf8004d90 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c160ec0 con 0x7f9cf8003f80 2023-11-28T11:12:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf800ba10 con 0x7f9d0c06ea70 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9cf8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0affd700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf8003f80 0x7f9cf8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0b7fe700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 0x7f9cf8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 -- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 msgr2=0x7f9cf8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c06ea70 0x7f9cf8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 -- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf8003f80 msgr2=0x7f9cf8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf8003f80 0x7f9cf8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 -- 172.21.15.83:0/14225844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80035b0 con 0x7f9cf8004d90 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0bfff700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9cf8005200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009fc0 tx=0x7f9cf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a1a956c47e831 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400ec30 con 0x7f9cf8004d90 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0400f040 con 0x7f9cf8004d90 2023-11-28T11:12:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.370+0000 7f9d0a7fc700 1 -- 172.21.15.83:0/14225844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04014480 con 0x7f9cf8004d90 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d11f16700 1 -- 172.21.15.83:0/14225844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 msgr2=0x7f9cf8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d11f16700 1 --2- 172.21.15.83:0/14225844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9cf8005200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009fc0 tx=0x7f9cf8005880 comp rx=0 tx=0).stop 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d0bfff700 1 -- 172.21.15.83:0/14225844 reap_dead start 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d11f16700 1 -- 172.21.15.83:0/14225844 shutdown_connections 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d11f16700 1 -- 172.21.15.83:0/14225844 >> 172.21.15.83:0/14225844 conn(0x7f9d0c0b2b80 msgr2=0x7f9d0c0b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d11f16700 1 -- 172.21.15.83:0/14225844 shutdown_connections 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.371+0000 7f9d11f16700 1 -- 172.21.15.83:0/14225844 wait complete. 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.372+0000 7f9d11f16700 1 Processor -- start 2023-11-28T11:12:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.372+0000 7f9d11f16700 1 -- start start 2023-11-28T11:12:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d11f16700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9d0c06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d11f16700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d0c0c3720 0x7f9d0c0c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d11f16700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c159f50 0x7f9d0c15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d11f16700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c15a900 con 0x7f9cf8004d90 2023-11-28T11:12:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d11f16700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c7430 con 0x7f9d0c0c3720 2023-11-28T11:12:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d11f16700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c4170 con 0x7f9d0c159f50 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d0b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9d0c06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d0b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9d0c06c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53410/0 (socket says 172.21.15.83:53410) 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.373+0000 7f9d0b7fe700 1 -- 172.21.15.83:0/1944512835 learned_addr learned my addr 172.21.15.83:0/1944512835 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d0b7fe700 1 -- 172.21.15.83:0/1944512835 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c159f50 msgr2=0x7f9d0c15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d0b7fe700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c159f50 0x7f9d0c15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d0b7fe700 1 -- 172.21.15.83:0/1944512835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d0c0c3720 msgr2=0x7f9d0c0c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d0b7fe700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d0c0c3720 0x7f9d0c0c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d0b7fe700 1 -- 172.21.15.83:0/1944512835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0d0540 con 0x7f9cf8004d90 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d0b7fe700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9d0c06c7d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400fb10 tx=0x7f9cf400fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40072e0 con 0x7f9cf8004d90 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf4007440 con 0x7f9cf8004d90 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c0d0260 con 0x7f9cf8004d90 2023-11-28T11:12:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4005830 con 0x7f9cf8004d90 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.374+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0d0820 con 0x7f9cf8004d90 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.375+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c0c7630 con 0x7f9cf8004d90 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.375+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9cf4005990 con 0x7f9cf8004d90 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.376+0000 7f9d03fff700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9cec0682f0 0x7f9cec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.376+0000 7f9d0affd700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9cec0682f0 0x7f9cec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.376+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9cf4086190 con 0x7f9cf8004d90 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.376+0000 7f9d0affd700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9cec0682f0 0x7f9cec06a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c09db70 tx=0x7f9cfc0093c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.380+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d0c0c7630 con 0x7f9cf8004d90 2023-11-28T11:12:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.562+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d0c0c7630 con 0x7f9cf8004d90 2023-11-28T11:12:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.563+0000 7f9d03fff700 1 -- 172.21.15.83:0/1944512835 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d0c0c7630 con 0x7f9cf8004d90 2023-11-28T11:12:34.564 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-28T11:12:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9cec0682f0 msgr2=0x7f9cec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9cec0682f0 0x7f9cec06a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c09db70 tx=0x7f9cfc0093c0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 msgr2=0x7f9d0c06c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9d0c06c7d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400fb10 tx=0x7f9cf400fed0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 shutdown_connections 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9d0c159f50 0x7f9d0c15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f9cec0682f0 0x7f9cec06a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf8004d90 0x7f9d0c06c7d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 --2- 172.21.15.83:0/1944512835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d0c0c3720 0x7f9d0c0c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 >> 172.21.15.83:0/1944512835 conn(0x7f9d0c0b2b80 msgr2=0x7f9d0c0ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 shutdown_connections 2023-11-28T11:12:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.565+0000 7f9d11f16700 1 -- 172.21.15.83:0/1944512835 wait complete. 2023-11-28T11:12:34.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.696+0000 7eff5e37f700 1 Processor -- start 2023-11-28T11:12:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.696+0000 7eff5e37f700 1 -- start start 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5e37f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 0x7eff5806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5e37f700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7eff580b18a0 0x7eff580b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5e37f700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff580cf540 con 0x7eff58159830 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5e37f700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5815d480 con 0x7eff580b18a0 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5e37f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b2260 con 0x7eff580cf0d0 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff577fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 0x7eff5806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5c91c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7eff580b18a0 0x7eff580b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff577fe700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 0x7eff5806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.83:37622/0 (socket says 172.21.15.83:37622) 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff577fe700 1 -- 172.21.15.83:0/3290024977 learned_addr learned my addr 172.21.15.83:0/3290024977 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff57fff700 1 --1- 172.21.15.83:0/3290024977 >> v1:172.21.15.90:6789/0 conn(0x7eff58159830 0x7eff5815de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.83:49624/0 (socket says 172.21.15.83:49624) 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5c91c700 1 -- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 msgr2=0x7eff5806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5c91c700 1 --2- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 0x7eff5806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5c91c700 1 -- 172.21.15.83:0/3290024977 >> v1:172.21.15.90:6789/0 conn(0x7eff58159830 legacy=0x7eff5815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff56ffd700 1 -- 172.21.15.83:0/3290024977 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683794233 0 0) 0x7eff580cf540 con 0x7eff58159830 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff5c91c700 1 -- 172.21.15.83:0/3290024977 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580cf790 con 0x7eff580b18a0 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.697+0000 7eff577fe700 1 --2- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 0x7eff5806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5c91c700 1 --2- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff580b1d20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff50009d90 tx=0x7eff5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f2da49bde96d28 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff56ffd700 1 -- 172.21.15.83:0/3290024977 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000e070 con 0x7eff580b18a0 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff56ffd700 1 -- 172.21.15.83:0/3290024977 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000bb20 con 0x7eff580b18a0 2023-11-28T11:12:34.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff56ffd700 1 -- 172.21.15.83:0/3290024977 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50012430 con 0x7eff580b18a0 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 -- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 msgr2=0x7eff580b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff580b1d20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff50009d90 tx=0x7eff5000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 -- 172.21.15.83:0/3290024977 shutdown_connections 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff580b1d20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3290024977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff580cf0d0 0x7eff5806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 -- 172.21.15.83:0/3290024977 >> 172.21.15.83:0/3290024977 conn(0x7eff5806e9a0 msgr2=0x7eff580cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 -- 172.21.15.83:0/3290024977 shutdown_connections 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.698+0000 7eff5e37f700 1 -- 172.21.15.83:0/3290024977 wait complete. 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.699+0000 7eff5e37f700 1 Processor -- start 2023-11-28T11:12:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 -- start start 2023-11-28T11:12:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff581e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 0x7eff581547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff58159830 0x7eff580b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff58154d20 con 0x7eff580cf0d0 2023-11-28T11:12:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5815ba10 con 0x7eff580b18a0 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.700+0000 7eff5e37f700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b11e0 con 0x7eff58159830 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 0x7eff581547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 0x7eff581547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53426/0 (socket says 172.21.15.83:53426) 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 -- 172.21.15.83:0/3717185116 learned_addr learned my addr 172.21.15.83:0/3717185116 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff57fff700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff581e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff5c91c700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff58159830 0x7eff580b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 -- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff58159830 msgr2=0x7eff580b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff58159830 0x7eff580b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 -- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 msgr2=0x7eff581e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff581e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 -- 172.21.15.83:0/3717185116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff48009550 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff5c91c700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff58159830 0x7eff580b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff57fff700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff581e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff577fe700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 0x7eff581547e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7eff4800ebe0 tx=0x7eff48009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4800c8b0 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4800ca10 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff50009a70 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4800cd80 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.701+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5815d690 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.702+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff580cfa70 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.703+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff48019030 con 0x7eff580cf0d0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.703+0000 7eff54ff9700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff440682f0 0x7eff4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.703+0000 7eff57fff700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff440682f0 0x7eff4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.703+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7eff48085e80 con 0x7eff580cf0d0 2023-11-28T11:12:34.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.703+0000 7eff57fff700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff440682f0 0x7eff4406a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000f60 tx=0x7eff4c007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:34.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.707+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff48052560 con 0x7eff580cf0d0 2023-11-28T11:12:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.933+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff5815d970 con 0x7eff580cf0d0 2023-11-28T11:12:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.934+0000 7eff54ff9700 1 -- 172.21.15.83:0/3717185116 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7eff48021090 con 0x7eff580cf0d0 2023-11-28T11:12:34.935 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:40.850456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff440682f0 msgr2=0x7eff4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff440682f0 0x7eff4406a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000f60 tx=0x7eff4c007ca0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 msgr2=0x7eff581547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 0x7eff581547e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7eff4800ebe0 tx=0x7eff48009e70 comp rx=0 tx=0).stop 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 shutdown_connections 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7eff58159830 0x7eff580b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff580b18a0 0x7eff581e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7eff440682f0 0x7eff4406a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 --2- 172.21.15.83:0/3717185116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff580cf0d0 0x7eff581547e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 >> 172.21.15.83:0/3717185116 conn(0x7eff5806e9a0 msgr2=0x7eff58157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 shutdown_connections 2023-11-28T11:12:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:34.936+0000 7eff5e37f700 1 -- 172.21.15.83:0/3717185116 wait complete. 2023-11-28T11:12:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2023-11-28T11:12:34.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.068+0000 7f908f4a7700 1 Processor -- start 2023-11-28T11:12:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.068+0000 7f908f4a7700 1 -- start start 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908f4a7700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f908815cd10 0x7f90880ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908f4a7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90880cbfb0 con 0x7f908815d0e0 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908f4a7700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90880cb710 con 0x7f90880b1fb0 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908f4a7700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90880cb9f0 con 0x7f908815cd10 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908ca42700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f908815d0e0 0x7f90880b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51292/0 (socket says 172.21.15.83:51292) 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908ca42700 1 -- 172.21.15.83:0/1994561933 learned_addr learned my addr 172.21.15.83:0/1994561933 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:35.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525197036 0 0) 0x7f90880cbfb0 con 0x7f908815d0e0 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90740035b0 con 0x7f908815d0e0 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908d243700 1 --2- 172.21.15.83:0/1994561933 >> v2:172.21.15.90:3300/0 conn(0x7f908815cd10 0x7f90880ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2647427311 0 0) 0x7f90740035b0 con 0x7f908815d0e0 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 >> v2:172.21.15.90:3300/0 conn(0x7f908815cd10 msgr2=0x7f90880ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 --2- 172.21.15.83:0/1994561933 >> v2:172.21.15.90:3300/0 conn(0x7f908815cd10 0x7f90880ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 >> v1:172.21.15.90:6789/0 conn(0x7f90880b1fb0 legacy=0x7f908806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880cbcd0 con 0x7f908815d0e0 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f9084001670 con 0x7f908815d0e0 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 >> v1:172.21.15.83:6789/0 conn(0x7f908815d0e0 legacy=0x7f90880b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9074003a00 0x7f9074003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9074004850 0x7f9074004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074005200 0x7f907400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880cb9f0 con 0x7f9074003a00 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907400c080 con 0x7f9074005200 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907400c240 con 0x7f9074004850 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908d243700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9074003a00 0x7f9074003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908da44700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074005200 0x7f907400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.069+0000 7f908ca42700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9074004850 0x7f9074004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908d243700 1 -- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9074004850 msgr2=0x7f9074004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908d243700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9074004850 0x7f9074004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908d243700 1 -- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074005200 msgr2=0x7f907400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908d243700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074005200 0x7f907400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908d243700 1 -- 172.21.15.83:0/1994561933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740035b0 con 0x7f9074003a00 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908da44700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074005200 0x7f907400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908da44700 1 -- 172.21.15.83:0/1994561933 reap_dead start 2023-11-28T11:12:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908d243700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9074003a00 0x7f9074003e10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9078009b60 tx=0x7f9074005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ba634746b00d40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907800e6c0 con 0x7f9074003a00 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907800e820 con 0x7f9074003a00 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f907ffff700 1 -- 172.21.15.83:0/1994561933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907800eb90 con 0x7f9074003a00 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 -- 172.21.15.83:0/1994561933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9074003a00 msgr2=0x7f9074003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9074003a00 0x7f9074003e10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9078009b60 tx=0x7f9074005880 comp rx=0 tx=0).stop 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 -- 172.21.15.83:0/1994561933 shutdown_connections 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 --2- 172.21.15.83:0/1994561933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9074003a00 0x7f9074003e10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 -- 172.21.15.83:0/1994561933 >> 172.21.15.83:0/1994561933 conn(0x7f908806e9a0 msgr2=0x7f90880cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 -- 172.21.15.83:0/1994561933 shutdown_connections 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.070+0000 7f908f4a7700 1 -- 172.21.15.83:0/1994561933 wait complete. 2023-11-28T11:12:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 Processor -- start 2023-11-28T11:12:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 -- start start 2023-11-28T11:12:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074003a00 0x7f90881f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f90880c4fe0 0x7f9088148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 0x7f9088148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880ae820 con 0x7f90881487a0 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880ae540 con 0x7f9074003a00 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908f4a7700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880c4500 con 0x7f90880c4fe0 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.072+0000 7f908da44700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 0x7f9088148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 0x7f9088148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53448/0 (socket says 172.21.15.83:53448) 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 -- 172.21.15.83:0/3359123525 learned_addr learned my addr 172.21.15.83:0/3359123525 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908d243700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074003a00 0x7f90881f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908ca42700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f90880c4fe0 0x7f9088148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 -- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f90880c4fe0 msgr2=0x7f9088148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f90880c4fe0 0x7f9088148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 -- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074003a00 msgr2=0x7f90881f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074003a00 0x7f90881f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 -- 172.21.15.83:0/3359123525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908806b660 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908da44700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 0x7f9088148c40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9088072970 tx=0x7f908000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908000eee0 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9080003c10 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90800050d0 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908806b9b0 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.073+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90880cd9f0 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.074+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90880c4340 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.075+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9080005230 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.075+0000 7f907dffb700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f906c0682f0 0x7f906c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.075+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9080085db0 con 0x7f90881487a0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.075+0000 7f908d243700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f906c0682f0 0x7f906c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.075+0000 7f908d243700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f906c0682f0 0x7f906c06a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f90780090b0 tx=0x7f907801e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.079+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9080052490 con 0x7f90881487a0 2023-11-28T11:12:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.262+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90880b0230 con 0x7f90881487a0 2023-11-28T11:12:35.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.264+0000 7f907dffb700 1 -- 172.21.15.83:0/3359123525 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18175 (secure 0 0 0) 0x7f9080055ab0 con 0x7f90881487a0 2023-11-28T11:12:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:35.266 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:12:25.948014+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:35.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f906c0682f0 msgr2=0x7f906c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f906c0682f0 0x7f906c06a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f90780090b0 tx=0x7f907801e040 comp rx=0 tx=0).stop 2023-11-28T11:12:35.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 msgr2=0x7f9088148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 0x7f9088148c40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9088072970 tx=0x7f908000ed50 comp rx=0 tx=0).stop 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 shutdown_connections 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f906c0682f0 0x7f906c06a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90881487a0 0x7f9088148c40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074003a00 0x7f90881f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 --2- 172.21.15.83:0/3359123525 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f90880c4fe0 0x7f9088148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 >> 172.21.15.83:0/3359123525 conn(0x7f908806e9a0 msgr2=0x7f90880afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 shutdown_connections 2023-11-28T11:12:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.266+0000 7f908f4a7700 1 -- 172.21.15.83:0/3359123525 wait complete. 2023-11-28T11:12:35.279 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:12:35.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.398+0000 7f4011de6700 1 Processor -- start 2023-11-28T11:12:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.398+0000 7f4011de6700 1 -- start start 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f4011de6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f400c0b1fb0 0x7f400c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f4011de6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c0cbfb0 con 0x7f400c15d0e0 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f4011de6700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c0cb710 con 0x7f400c15cd10 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f4011de6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0cb9f0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f400c0b1fb0 0x7f400c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f400c0b1fb0 0x7f400c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53456/0 (socket says 172.21.15.83:53456) 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400bfff700 1 -- 172.21.15.83:0/2468056868 learned_addr learned my addr 172.21.15.83:0/2468056868 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400affd700 1 --1- 172.21.15.83:0/2468056868 >> v1:172.21.15.83:6789/0 conn(0x7f400c15d0e0 0x7f400c06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51314/0 (socket says 172.21.15.83:51314) 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400bfff700 1 -- 172.21.15.83:0/2468056868 >> v1:172.21.15.90:6789/0 conn(0x7f400c15cd10 legacy=0x7f400c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400bfff700 1 -- 172.21.15.83:0/2468056868 >> v1:172.21.15.83:6789/0 conn(0x7f400c15d0e0 legacy=0x7f400c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400bfff700 1 -- 172.21.15.83:0/2468056868 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c0cbcd0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.399+0000 7f400a7fc700 1 -- 172.21.15.83:0/2468056868 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422697096 0 0) 0x7f400c0cbfb0 con 0x7f400c15d0e0 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f400bfff700 1 --2- 172.21.15.83:0/2468056868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0ca970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4004009e50 tx=0x7f400400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393758575c350dc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f400a7fc700 1 -- 172.21.15.83:0/2468056868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400400e070 con 0x7f400c0b1fb0 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f400a7fc700 1 -- 172.21.15.83:0/2468056868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4004003a20 con 0x7f400c0b1fb0 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f400a7fc700 1 -- 172.21.15.83:0/2468056868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004003dd0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 -- 172.21.15.83:0/2468056868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 msgr2=0x7f400c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 --2- 172.21.15.83:0/2468056868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0ca970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4004009e50 tx=0x7f400400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 -- 172.21.15.83:0/2468056868 shutdown_connections 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 --2- 172.21.15.83:0/2468056868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0ca970 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 -- 172.21.15.83:0/2468056868 >> 172.21.15.83:0/2468056868 conn(0x7f400c06e9a0 msgr2=0x7f400c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 -- 172.21.15.83:0/2468056868 shutdown_connections 2023-11-28T11:12:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.400+0000 7f4011de6700 1 -- 172.21.15.83:0/2468056868 wait complete. 2023-11-28T11:12:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 Processor -- start 2023-11-28T11:12:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 -- start start 2023-11-28T11:12:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f400c15cd10 0x7f400c0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f400c15d0e0 0x7f400c0ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0c3920 con 0x7f400c0b1fb0 2023-11-28T11:12:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0c3ee0 con 0x7f400c15cd10 2023-11-28T11:12:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.402+0000 7f4011de6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1e70b0 con 0x7f400c15d0e0 2023-11-28T11:12:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53458/0 (socket says 172.21.15.83:53458) 2023-11-28T11:12:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 -- 172.21.15.83:0/2925581587 learned_addr learned my addr 172.21.15.83:0/2925581587 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 -- 172.21.15.83:0/2925581587 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f400c15d0e0 msgr2=0x7f400c0ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f400c15d0e0 0x7f400c0ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 -- 172.21.15.83:0/2925581587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f400c15cd10 msgr2=0x7f400c0c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f400c15cd10 0x7f400c0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 -- 172.21.15.83:0/2925581587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009b30 con 0x7f400c0b1fb0 2023-11-28T11:12:35.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f400b7fe700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0c5a70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00c4d0 tx=0x7f3ffc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0054e0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc003680 con 0x7f400c0b1fb0 2023-11-28T11:12:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c06b660 con 0x7f400c0b1fb0 2023-11-28T11:12:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc003a30 con 0x7f400c0b1fb0 2023-11-28T11:12:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.403+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c06b9b0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.404+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c0cd9f0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.404+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ffc019030 con 0x7f400c0b1fb0 2023-11-28T11:12:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.405+0000 7f4003fff700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3fec0682f0 0x7f3fec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.405+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3ffc085200 con 0x7f400c0b1fb0 2023-11-28T11:12:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.405+0000 7f400affd700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3fec0682f0 0x7f3fec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.405+0000 7f400affd700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3fec0682f0 0x7f3fec06a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f400c0b00b0 tx=0x7f3ff40079f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.409+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ffc0518e0 con 0x7f400c0b1fb0 2023-11-28T11:12:35.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.633+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f400c074320 con 0x7f400c0b1fb0 2023-11-28T11:12:35.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.634+0000 7f4003fff700 1 -- 172.21.15.83:0/2925581587 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f3ffc054f00 con 0x7f400c0b1fb0 2023-11-28T11:12:35.635 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:35.636 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:40.850456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3fec0682f0 msgr2=0x7f3fec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3fec0682f0 0x7f3fec06a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f400c0b00b0 tx=0x7f3ff40079f0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 msgr2=0x7f400c0c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0c5a70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00c4d0 tx=0x7f3ffc00c890 comp rx=0 tx=0).stop 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 shutdown_connections 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f400c15d0e0 0x7f400c0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f3fec0682f0 0x7f3fec06a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f400c0b1fb0 0x7f400c0c5a70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 --2- 172.21.15.83:0/2925581587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f400c15cd10 0x7f400c0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 >> 172.21.15.83:0/2925581587 conn(0x7f400c06e9a0 msgr2=0x7f400c0c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 shutdown_connections 2023-11-28T11:12:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.636+0000 7f4011de6700 1 -- 172.21.15.83:0/2925581587 wait complete. 2023-11-28T11:12:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2023-11-28T11:12:35.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:35.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.768+0000 7f6fef7b3700 1 Processor -- start 2023-11-28T11:12:35.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.768+0000 7f6fef7b3700 1 -- start start 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fef7b3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe815cd10 0x7f6fe80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fef7b3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6fe815d0e0 0x7f6fe80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fef7b3700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6fe80b18a0 0x7f6fe80b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fef7b3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe80b2280 con 0x7f6fe815d0e0 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fef7b3700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe80cbfb0 con 0x7f6fe80b18a0 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fef7b3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80cb4e0 con 0x7f6fe815cd10 2023-11-28T11:12:35.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6fe815d0e0 0x7f6fe80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6fe815d0e0 0x7f6fe80cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53466/0 (socket says 172.21.15.83:53466) 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2583477019 learned_addr learned my addr 172.21.15.83:0/2583477019 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fed54f700 1 --2- 172.21.15.83:0/2583477019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe815cd10 0x7f6fe80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fedd50700 1 --2- 172.21.15.83:0/2583477019 >> v2:172.21.15.90:3300/0 conn(0x7f6fe80b18a0 0x7f6fe80b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2583477019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe815cd10 msgr2=0x7f6fe80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 --2- 172.21.15.83:0/2583477019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe815cd10 0x7f6fe80ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2583477019 >> v2:172.21.15.90:3300/0 conn(0x7f6fe80b18a0 msgr2=0x7f6fe80b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 --2- 172.21.15.83:0/2583477019 >> v2:172.21.15.90:3300/0 conn(0x7f6fe80b18a0 0x7f6fe80b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2583477019 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe80cb710 con 0x7f6fe815d0e0 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fedd50700 1 --2- 172.21.15.83:0/2583477019 >> v2:172.21.15.90:3300/0 conn(0x7f6fe80b18a0 0x7f6fe80b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.769+0000 7f6fecd4e700 1 --2- 172.21.15.83:0/2583477019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80cafa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8160910 tx=0x7f6fd8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b005c556d5a951e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fdffff700 1 -- 172.21.15.83:0/2583477019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8011830 con 0x7f6fe815d0e0 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fdffff700 1 -- 172.21.15.83:0/2583477019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd8011990 con 0x7f6fe815d0e0 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fdffff700 1 -- 172.21.15.83:0/2583477019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8011d20 con 0x7f6fe815d0e0 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2583477019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815d0e0 msgr2=0x7f6fe80cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2583477019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80cafa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8160910 tx=0x7f6fd8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2583477019 shutdown_connections 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2583477019 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe815cd10 0x7f6fe80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2583477019 >> v2:172.21.15.90:3300/0 conn(0x7f6fe80b18a0 0x7f6fe80b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2583477019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80cafa0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2583477019 >> 172.21.15.83:0/2583477019 conn(0x7f6fe806e9a0 msgr2=0x7f6fe80cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:35.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2583477019 shutdown_connections 2023-11-28T11:12:35.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.770+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2583477019 wait complete. 2023-11-28T11:12:35.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.771+0000 7f6fef7b3700 1 Processor -- start 2023-11-28T11:12:35.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.771+0000 7f6fef7b3700 1 -- start start 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fef7b3700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe80b18a0 0x7f6fe80c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fef7b3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 0x7f6fe81509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fef7b3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fef7b3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80c8130 con 0x7f6fe815cd10 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fef7b3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80c8410 con 0x7f6fe815d0e0 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fef7b3700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80c86f0 con 0x7f6fe80b18a0 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fecd4e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 0x7f6fe81509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fecd4e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 0x7f6fe81509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53472/0 (socket says 172.21.15.83:53472) 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.772+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2749029044 learned_addr learned my addr 172.21.15.83:0/2749029044 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe80b18a0 msgr2=0x7f6fe80c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fed54f700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe80b18a0 0x7f6fe80c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fecd4e700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe80b18a0 0x7f6fe80c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fedd50700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fe815d0e0 msgr2=0x7f6fe80ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fecd4e700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fecd4e700 1 -- 172.21.15.83:0/2749029044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8009550 con 0x7f6fe815cd10 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fed54f700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe80b18a0 0x7f6fe80c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fecd4e700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 0x7f6fe81509e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009c10 tx=0x7f6fd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80036a0 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd801e070 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe80cbcd0 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003a60 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.773+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe80cb9f0 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.774+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe806b9b0 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.774+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6fd800f050 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.775+0000 7f6fddffb700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fcc0682f0 0x7f6fcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.775+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6fd8086d40 con 0x7f6fe815cd10 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.775+0000 7f6fed54f700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fcc0682f0 0x7f6fcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.775+0000 7f6fed54f700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fcc0682f0 0x7f6fcc06a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009b60 tx=0x7f6fe4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:35.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:35.779+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6fd8053420 con 0x7f6fe815cd10 2023-11-28T11:12:36.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.003+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fe8074320 con 0x7f6fe815cd10 2023-11-28T11:12:36.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.004+0000 7f6fddffb700 1 -- 172.21.15.83:0/2749029044 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f6fd8020090 con 0x7f6fe815cd10 2023-11-28T11:12:36.006 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:36.006 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:11:24.379444+0000","modified":"2023-11-28T11:11:40.850456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.83:6837/3038492821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3038492821},{"type":"v1","addr":"172.21.15.83:6837","nonce":3038492821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:36.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fcc0682f0 msgr2=0x7f6fcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fcc0682f0 0x7f6fcc06a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009b60 tx=0x7f6fe4008040 comp rx=0 tx=0).stop 2023-11-28T11:12:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 msgr2=0x7f6fe81509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 0x7f6fe81509e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009c10 tx=0x7f6fd8004750 comp rx=0 tx=0).stop 2023-11-28T11:12:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 shutdown_connections 2023-11-28T11:12:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fe815d0e0 0x7f6fe80ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6fe80b18a0 0x7f6fe80c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f6fcc0682f0 0x7f6fcc06a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 --2- 172.21.15.83:0/2749029044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe815cd10 0x7f6fe81509e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 >> 172.21.15.83:0/2749029044 conn(0x7f6fe806e9a0 msgr2=0x7f6fe80cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 shutdown_connections 2023-11-28T11:12:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.006+0000 7f6fef7b3700 1 -- 172.21.15.83:0/2749029044 wait complete. 2023-11-28T11:12:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2023-11-28T11:12:36.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:36.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.138+0000 7f1c4a3eb700 1 Processor -- start 2023-11-28T11:12:36.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.138+0000 7f1c4a3eb700 1 -- start start 2023-11-28T11:12:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c4a3eb700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1c4415cd10 0x7f1c440ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c4a3eb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c4415d0e0 0x7f1c440cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c4a3eb700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c440cbfb0 con 0x7f1c440b18a0 2023-11-28T11:12:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c4a3eb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4406cee0 con 0x7f1c4415d0e0 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c4a3eb700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c440cb710 con 0x7f1c4415cd10 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c437fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c4415d0e0 0x7f1c440cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c437fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c4415d0e0 0x7f1c440cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53482/0 (socket says 172.21.15.83:53482) 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.139+0000 7f1c437fe700 1 -- 172.21.15.83:0/75888931 learned_addr learned my addr 172.21.15.83:0/75888931 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c437fe700 1 -- 172.21.15.83:0/75888931 >> v2:172.21.15.90:3300/0 conn(0x7f1c4415cd10 msgr2=0x7f1c440ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c43fff700 1 --2- 172.21.15.83:0/75888931 >> v2:172.21.15.90:3300/0 conn(0x7f1c4415cd10 0x7f1c440ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c437fe700 1 --2- 172.21.15.83:0/75888931 >> v2:172.21.15.90:3300/0 conn(0x7f1c4415cd10 0x7f1c440ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c437fe700 1 -- 172.21.15.83:0/75888931 >> v1:172.21.15.90:6789/0 conn(0x7f1c440b18a0 legacy=0x7f1c441609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c437fe700 1 -- 172.21.15.83:0/75888931 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440cb4e0 con 0x7f1c4415d0e0 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c437fe700 1 --2- 172.21.15.83:0/75888931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4415d0e0 0x7f1c440cafa0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1c440716f0 tx=0x7f1c34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fba1699855ccfb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c42ffd700 1 -- 172.21.15.83:0/75888931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34004230 con 0x7f1c4415d0e0 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c42ffd700 1 -- 172.21.15.83:0/75888931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c34004390 con 0x7f1c4415d0e0 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c42ffd700 1 -- 172.21.15.83:0/75888931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340036a0 con 0x7f1c4415d0e0 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/75888931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4415d0e0 msgr2=0x7f1c440cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/75888931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4415d0e0 0x7f1c440cafa0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1c440716f0 tx=0x7f1c34009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/75888931 shutdown_connections 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/75888931 >> v2:172.21.15.90:3300/0 conn(0x7f1c4415cd10 0x7f1c440ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/75888931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4415d0e0 0x7f1c440cafa0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.140+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/75888931 >> 172.21.15.83:0/75888931 conn(0x7f1c4406e9a0 msgr2=0x7f1c440cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.141+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/75888931 shutdown_connections 2023-11-28T11:12:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.141+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/75888931 wait complete. 2023-11-28T11:12:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.142+0000 7f1c4a3eb700 1 Processor -- start 2023-11-28T11:12:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.142+0000 7f1c4a3eb700 1 -- start start 2023-11-28T11:12:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.142+0000 7f1c4a3eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 0x7f1c4415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c4415cd10 0x7f1c441f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c4415d0e0 0x7f1c4414b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c44160530 con 0x7f1c440b18a0 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4414b2f0 con 0x7f1c4415cd10 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415fc90 con 0x7f1c4415d0e0 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 0x7f1c4415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 0x7f1c4415f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53486/0 (socket says 172.21.15.83:53486) 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 -- 172.21.15.83:0/1343594005 learned_addr learned my addr 172.21.15.83:0/1343594005 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 -- 172.21.15.83:0/1343594005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c4415d0e0 msgr2=0x7f1c4414b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c4415d0e0 0x7f1c4414b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 -- 172.21.15.83:0/1343594005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c4415cd10 msgr2=0x7f1c441f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c4415cd10 0x7f1c441f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 -- 172.21.15.83:0/1343594005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c34009550 con 0x7f1c440b18a0 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c43fff700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 0x7f1c4415f420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c820 tx=0x7f1c3800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38011800 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38011960 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c440cbcd0 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38011cf0 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.143+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c440cb9f0 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.144+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4406b9b0 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.144+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c38021070 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.145+0000 7f1c40ff9700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c300682f0 0x7f1c3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.145+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1c38085d60 con 0x7f1c440b18a0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.145+0000 7f1c437fe700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c300682f0 0x7f1c3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.145+0000 7f1c437fe700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c300682f0 0x7f1c3006a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c44147b50 tx=0x7f1c34010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.149+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c380523b0 con 0x7f1c440b18a0 2023-11-28T11:12:36.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.334+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c44074320 con 0x7f1c440b18a0 2023-11-28T11:12:36.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.335+0000 7f1c40ff9700 1 -- 172.21.15.83:0/1343594005 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18175 (secure 0 0 0) 0x7f1c3801f070 con 0x7f1c440b18a0 2023-11-28T11:12:36.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:36.337 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:12:25.948014+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:11:22.194488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:11:23.354896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c300682f0 msgr2=0x7f1c3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c300682f0 0x7f1c3006a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c44147b50 tx=0x7f1c34010000 comp rx=0 tx=0).stop 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 msgr2=0x7f1c4415f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 0x7f1c4415f420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c820 tx=0x7f1c3800cbe0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 shutdown_connections 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c4415d0e0 0x7f1c4414b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1c300682f0 0x7f1c3006a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c440b18a0 0x7f1c4415f420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 --2- 172.21.15.83:0/1343594005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c4415cd10 0x7f1c441f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 >> 172.21.15.83:0/1343594005 conn(0x7f1c4406e9a0 msgr2=0x7f1c4415c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 shutdown_connections 2023-11-28T11:12:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.337+0000 7f1c4a3eb700 1 -- 172.21.15.83:0/1343594005 wait complete. 2023-11-28T11:12:36.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:12:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.472+0000 7fdccc085700 1 Processor -- start 2023-11-28T11:12:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.472+0000 7fdccc085700 1 -- start start 2023-11-28T11:12:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdccc085700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdcc40aee20 0x7fdcc40b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdccc085700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdccc085700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc40b5b00 con 0x7fdcc406c370 2023-11-28T11:12:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdccc085700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc406c800 con 0x7fdcc40aee20 2023-11-28T11:12:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdccc085700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc406cae0 con 0x7fdcc40af1f0 2023-11-28T11:12:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdcc9e21700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdcc40aee20 0x7fdcc40b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdcc9620700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdcc406c370 0x7fdcc406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51354/0 (socket says 172.21.15.83:51354) 2023-11-28T11:12:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.473+0000 7fdcc9620700 1 -- 172.21.15.83:0/2964009628 learned_addr learned my addr 172.21.15.83:0/2964009628 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcca622700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc9e21700 1 -- 172.21.15.83:0/2964009628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 msgr2=0x7fdcc40cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc9e21700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc9e21700 1 -- 172.21.15.83:0/2964009628 >> v1:172.21.15.83:6789/0 conn(0x7fdcc406c370 legacy=0x7fdcc406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc9e21700 1 -- 172.21.15.83:0/2964009628 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc406cdc0 con 0x7fdcc40aee20 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc8e1f700 1 -- 172.21.15.83:0/2964009628 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480885685 0 0) 0x7fdcc0001110 con 0x7fdcc406c370 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcca622700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc9e21700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc40b55c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009820 tx=0x7fdcb4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436c1b814f54420b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc8e1f700 1 -- 172.21.15.83:0/2964009628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4005e30 con 0x7fdcc40aee20 2023-11-28T11:12:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc8e1f700 1 -- 172.21.15.83:0/2964009628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4006030 con 0x7fdcc40aee20 2023-11-28T11:12:36.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.474+0000 7fdcc8e1f700 1 -- 172.21.15.83:0/2964009628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40063e0 con 0x7fdcc40aee20 2023-11-28T11:12:36.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 -- 172.21.15.83:0/2964009628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 msgr2=0x7fdcc40b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc40b55c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009820 tx=0x7fdcb4009b30 comp rx=0 tx=0).stop 2023-11-28T11:12:36.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 -- 172.21.15.83:0/2964009628 shutdown_connections 2023-11-28T11:12:36.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 --2- 172.21.15.83:0/2964009628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc40b55c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 -- 172.21.15.83:0/2964009628 >> 172.21.15.83:0/2964009628 conn(0x7fdcc40b2b60 msgr2=0x7fdcc40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 -- 172.21.15.83:0/2964009628 shutdown_connections 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.475+0000 7fdccc085700 1 -- 172.21.15.83:0/2964009628 wait complete. 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.476+0000 7fdccc085700 1 Processor -- start 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.476+0000 7fdccc085700 1 -- start start 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc406c370 0x7fdcc40af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc41e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41e6c30 con 0x7fdcc40aee20 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4140190 con 0x7fdcc406c370 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4140470 con 0x7fdcc40af1f0 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc41e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc41e66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53502/0 (socket says 172.21.15.83:53502) 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 -- 172.21.15.83:0/902992501 learned_addr learned my addr 172.21.15.83:0/902992501 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:36.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcca622700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9e21700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc406c370 0x7fdcc40af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 -- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 msgr2=0x7fdcc40adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 -- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc406c370 msgr2=0x7fdcc40af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc406c370 0x7fdcc40af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 -- 172.21.15.83:0/902992501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4009430 con 0x7fdcc40aee20 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcc9620700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc41e66f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000c550 tx=0x7fdcc000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0004c80 con 0x7fdcc40aee20 2023-11-28T11:12:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc0004e00 con 0x7fdcc40aee20 2023-11-28T11:12:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc406fc70 con 0x7fdcc40aee20 2023-11-28T11:12:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0007310 con 0x7fdcc40aee20 2023-11-28T11:12:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.477+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc406d0a0 con 0x7fdcc40aee20 2023-11-28T11:12:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.478+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc406d380 con 0x7fdcc40aee20 2023-11-28T11:12:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.479+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdcc001a020 con 0x7fdcc40aee20 2023-11-28T11:12:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.479+0000 7fdcba7fc700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdca80682f0 0x7fdca806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.479+0000 7fdcc9e21700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdca80682f0 0x7fdca806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.479+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fdcc0085940 con 0x7fdcc40aee20 2023-11-28T11:12:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.480+0000 7fdcc9e21700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdca80682f0 0x7fdca806a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009b70 tx=0x7fdcb4008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.484+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdcc0052020 con 0x7fdcc40aee20 2023-11-28T11:12:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.707+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdcc40b5b00 con 0x7fdcc40aee20 2023-11-28T11:12:36.984 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-28T11:12:36.983+0000 7fdf5c1f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:12:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.989+0000 7fdcba7fc700 1 -- 172.21.15.83:0/902992501 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdcc40b5b00 con 0x7fdcc40aee20 2023-11-28T11:12:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdca80682f0 msgr2=0x7fdca806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdca80682f0 0x7fdca806a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009b70 tx=0x7fdcb4008070 comp rx=0 tx=0).stop 2023-11-28T11:12:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 msgr2=0x7fdcc41e66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc41e66f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000c550 tx=0x7fdcc000c910 comp rx=0 tx=0).stop 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 shutdown_connections 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcc40af1f0 0x7fdcc40adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc406c370 0x7fdcc40af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdca80682f0 0x7fdca806a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 --2- 172.21.15.83:0/902992501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40aee20 0x7fdcc41e66f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 >> 172.21.15.83:0/902992501 conn(0x7fdcc40b2b60 msgr2=0x7fdcc40cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 shutdown_connections 2023-11-28T11:12:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:36.992+0000 7fdccc085700 1 -- 172.21.15.83:0/902992501 wait complete. 2023-11-28T11:12:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:12:36.997 INFO:tasks.ceph.mds.c.smithi083.stderr: -7309> 2023-11-28T11:11:29.974+0000 7f20ad068700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:12:36.999 INFO:tasks.ceph.mds.c.smithi083.stderr: -3701> 2023-11-28T11:11:37.328+0000 7f20a9861700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5884=0-4194304@1} caps={5884=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5884 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562f95c13b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:12:37.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:12:37.028 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-28T11:12:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.140+0000 7fef2a88b700 1 Processor -- start 2023-11-28T11:12:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.140+0000 7fef2a88b700 1 -- start start 2023-11-28T11:12:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef2a88b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef240c99b0 0x7fef241599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef2a88b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2406b1e0 con 0x7fef2415d9c0 2023-11-28T11:12:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef2a88b700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2415de40 con 0x7fef240cc020 2023-11-28T11:12:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef2a88b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef240ceda0 con 0x7fef240c99b0 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef237fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef240c99b0 0x7fef241599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef237fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef240c99b0 0x7fef241599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53526/0 (socket says 172.21.15.83:53526) 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef237fe700 1 -- 172.21.15.83:0/1736685405 learned_addr learned my addr 172.21.15.83:0/1736685405 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.141+0000 7fef28e28700 1 --1- 172.21.15.83:0/1736685405 >> v1:172.21.15.83:6789/0 conn(0x7fef2415d9c0 0x7fef2406b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51388/0 (socket says 172.21.15.83:51388) 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef237fe700 1 -- 172.21.15.83:0/1736685405 >> v1:172.21.15.90:6789/0 conn(0x7fef240cc020 legacy=0x7fef240cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef237fe700 1 -- 172.21.15.83:0/1736685405 >> v1:172.21.15.83:6789/0 conn(0x7fef2415d9c0 legacy=0x7fef2406b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef237fe700 1 -- 172.21.15.83:0/1736685405 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2415a010 con 0x7fef240c99b0 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef22ffd700 1 -- 172.21.15.83:0/1736685405 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49798006 0 0) 0x7fef2406b1e0 con 0x7fef2415d9c0 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef237fe700 1 --2- 172.21.15.83:0/1736685405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef241599c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fef180098f0 tx=0x7fef18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef67da9e58e81506 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef22ffd700 1 -- 172.21.15.83:0/1736685405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800b8a0 con 0x7fef240c99b0 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef22ffd700 1 -- 172.21.15.83:0/1736685405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1800ba00 con 0x7fef240c99b0 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.142+0000 7fef22ffd700 1 -- 172.21.15.83:0/1736685405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800bd90 con 0x7fef240c99b0 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 -- 172.21.15.83:0/1736685405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 msgr2=0x7fef241599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 --2- 172.21.15.83:0/1736685405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef241599c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fef180098f0 tx=0x7fef18009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 -- 172.21.15.83:0/1736685405 shutdown_connections 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 --2- 172.21.15.83:0/1736685405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef241599c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 -- 172.21.15.83:0/1736685405 >> 172.21.15.83:0/1736685405 conn(0x7fef240b1090 msgr2=0x7fef240b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 -- 172.21.15.83:0/1736685405 shutdown_connections 2023-11-28T11:12:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.143+0000 7fef2a88b700 1 -- 172.21.15.83:0/1736685405 wait complete. 2023-11-28T11:12:37.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.144+0000 7fef2a88b700 1 Processor -- start 2023-11-28T11:12:37.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.144+0000 7fef2a88b700 1 -- start start 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef2a88b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef240c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef2a88b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef240cc020 0x7fef2415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef2a88b700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef2415d9c0 0x7fef240b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef2a88b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef240c7bb0 con 0x7fef240c99b0 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef2a88b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef240c7e50 con 0x7fef240cc020 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef2a88b700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef240c8130 con 0x7fef2415d9c0 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef240c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef240c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53530/0 (socket says 172.21.15.83:53530) 2023-11-28T11:12:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 -- 172.21.15.83:0/2747234673 learned_addr learned my addr 172.21.15.83:0/2747234673 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef237fe700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef240cc020 0x7fef2415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef28e28700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef2415d9c0 0x7fef240b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 -- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef2415d9c0 msgr2=0x7fef240b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef2415d9c0 0x7fef240b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 -- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef240cc020 msgr2=0x7fef2415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef240cc020 0x7fef2415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef23fff700 1 -- 172.21.15.83:0/2747234673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18009550 con 0x7fef240c99b0 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.145+0000 7fef237fe700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef240cc020 0x7fef2415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:37.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.146+0000 7fef23fff700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef240c8c10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c450 tx=0x7fef1400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.146+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef140052d0 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.146+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14005430 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.146+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2415a5d0 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.146+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef140038c0 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.146+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2415a2f0 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.147+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2415a8b0 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.147+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef14019030 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.147+0000 7fef20ff9700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef100682f0 0x7fef1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.147+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6526+0+0 (secure 0 0 0) 0x7fef14085050 con 0x7fef240c99b0 2023-11-28T11:12:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.147+0000 7fef237fe700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef100682f0 0x7fef1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.148+0000 7fef237fe700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef100682f0 0x7fef1006a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef18009130 tx=0x7fef1801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.152+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef140516e0 con 0x7fef240c99b0 2023-11-28T11:12:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.379+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fef240742b0 con 0x7fef240c99b0 2023-11-28T11:12:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.991+0000 7fef20ff9700 1 -- 172.21.15.83:0/2747234673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef14054d00 con 0x7fef240c99b0 2023-11-28T11:12:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef100682f0 msgr2=0x7fef1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef100682f0 0x7fef1006a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef18009130 tx=0x7fef1801c040 comp rx=0 tx=0).stop 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 msgr2=0x7fef240c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef240c8c10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c450 tx=0x7fef1400c810 comp rx=0 tx=0).stop 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 shutdown_connections 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fef2415d9c0 0x7fef240b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fef100682f0 0x7fef1006a7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef240c99b0 0x7fef240c8c10 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 --2- 172.21.15.83:0/2747234673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef240cc020 0x7fef2415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 >> 172.21.15.83:0/2747234673 conn(0x7fef240b1090 msgr2=0x7fef2415c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 shutdown_connections 2023-11-28T11:12:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:37.994+0000 7fef2a88b700 1 -- 172.21.15.83:0/2747234673 wait complete. 2023-11-28T11:12:38.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:12:38.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.176+0000 7f81e9df6700 1 Processor -- start 2023-11-28T11:12:38.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.176+0000 7f81e9df6700 1 -- start start 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e9df6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81e4159580 0x7f81e40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e9df6700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f81e4159950 0x7f81e4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e9df6700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e40cde60 con 0x7f81e406e920 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e9df6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40cced0 con 0x7f81e4159580 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e9df6700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e415a490 con 0x7f81e4159950 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81e4159580 0x7f81e40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81e4159580 0x7f81e40cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53540/0 (socket says 172.21.15.83:53540) 2023-11-28T11:12:38.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 -- 172.21.15.83:0/3298065262 learned_addr learned my addr 172.21.15.83:0/3298065262 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e2ffd700 1 --2- 172.21.15.83:0/3298065262 >> v2:172.21.15.90:3300/0 conn(0x7f81e4159950 0x7f81e4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 -- 172.21.15.83:0/3298065262 >> v2:172.21.15.90:3300/0 conn(0x7f81e4159950 msgr2=0x7f81e4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 --2- 172.21.15.83:0/3298065262 >> v2:172.21.15.90:3300/0 conn(0x7f81e4159950 0x7f81e4159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 -- 172.21.15.83:0/3298065262 >> v1:172.21.15.90:6789/0 conn(0x7f81e406e920 legacy=0x7f81e40c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.177+0000 7f81e37fe700 1 -- 172.21.15.83:0/3298065262 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e415a770 con 0x7f81e4159580 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e37fe700 1 --2- 172.21.15.83:0/3298065262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e4159580 0x7f81e40cd920 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f81e40cbce0 tx=0x7f81d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ab7eaed368edff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e27fc700 1 -- 172.21.15.83:0/3298065262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8004230 con 0x7f81e4159580 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e27fc700 1 -- 172.21.15.83:0/3298065262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8004390 con 0x7f81e4159580 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e27fc700 1 -- 172.21.15.83:0/3298065262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80036a0 con 0x7f81e4159580 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 -- 172.21.15.83:0/3298065262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e4159580 msgr2=0x7f81e40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3298065262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e4159580 0x7f81e40cd920 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f81e40cbce0 tx=0x7f81d8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 -- 172.21.15.83:0/3298065262 shutdown_connections 2023-11-28T11:12:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3298065262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e4159580 0x7f81e40cd920 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:38.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3298065262 >> v2:172.21.15.90:3300/0 conn(0x7f81e4159950 0x7f81e4159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:38.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 -- 172.21.15.83:0/3298065262 >> 172.21.15.83:0/3298065262 conn(0x7f81e40cf300 msgr2=0x7f81e40cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:38.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 -- 172.21.15.83:0/3298065262 shutdown_connections 2023-11-28T11:12:38.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.178+0000 7f81e9df6700 1 -- 172.21.15.83:0/3298065262 wait complete. 2023-11-28T11:12:38.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.179+0000 7f81e9df6700 1 Processor -- start 2023-11-28T11:12:38.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 -- start start 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 0x7f81e40c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81e4159580 0x7f81e4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4159950 0x7f81e4071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e4158160 con 0x7f81e406e920 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e41e2750 con 0x7f81e4159950 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.180+0000 7f81e9df6700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40c2df0 con 0x7f81e4159580 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 0x7f81e40c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 0x7f81e40c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53544/0 (socket says 172.21.15.83:53544) 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 -- 172.21.15.83:0/3440479541 learned_addr learned my addr 172.21.15.83:0/3440479541 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:38.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 -- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81e4159580 msgr2=0x7f81e4157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81e4159580 0x7f81e4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e3fff700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4159950 0x7f81e4071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 -- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4159950 msgr2=0x7f81e4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4159950 0x7f81e4071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 -- 172.21.15.83:0/3440479541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d8009550 con 0x7f81e406e920 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e37fe700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 0x7f81e40c3bb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009c10 tx=0x7f81d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8003d80 con 0x7f81e406e920 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d801f070 con 0x7f81e406e920 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40cd400 con 0x7f81e406e920 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8011870 con 0x7f81e406e920 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.181+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e40cd120 con 0x7f81e406e920 2023-11-28T11:12:38.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.182+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e40cec10 con 0x7f81e406e920 2023-11-28T11:12:38.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.183+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81d8017020 con 0x7f81e406e920 2023-11-28T11:12:38.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.183+0000 7f81cbfff700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81d00682f0 0x7f81d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:38.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.183+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f81d80873f0 con 0x7f81e406e920 2023-11-28T11:12:38.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.183+0000 7f81e2ffd700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81d00682f0 0x7f81d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:38.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.183+0000 7f81e2ffd700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81d00682f0 0x7f81d006a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81e40c9130 tx=0x7f81d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:38.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.187+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81d8053a20 con 0x7f81e406e920 2023-11-28T11:12:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:38.370+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81e40cde60 con 0x7f81e406e920 2023-11-28T11:12:39.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.048+0000 7f81cbfff700 1 -- 172.21.15.83:0/3440479541 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f81e40cde60 con 0x7f81e406e920 2023-11-28T11:12:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81d00682f0 msgr2=0x7f81d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81d00682f0 0x7f81d006a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81e40c9130 tx=0x7f81d4008040 comp rx=0 tx=0).stop 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 msgr2=0x7f81e40c3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 0x7f81e40c3bb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009c10 tx=0x7f81d8004750 comp rx=0 tx=0).stop 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 shutdown_connections 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4159950 0x7f81e4071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81d00682f0 0x7f81d006a7a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e406e920 0x7f81e40c3bb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 --2- 172.21.15.83:0/3440479541 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81e4159580 0x7f81e4157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 >> 172.21.15.83:0/3440479541 conn(0x7f81e40cf300 msgr2=0x7f81e40c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:39.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 shutdown_connections 2023-11-28T11:12:39.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.051+0000 7f81e9df6700 1 -- 172.21.15.83:0/3440479541 wait complete. 2023-11-28T11:12:39.054 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:12:39.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:12:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.189+0000 7f503217c700 1 Processor -- start 2023-11-28T11:12:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.189+0000 7f503217c700 1 -- start start 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.190+0000 7f503217c700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f502c06cf30 0x7f502c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.190+0000 7f503217c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c15d100 con 0x7f502c15cd30 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.190+0000 7f503217c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c06beb0 con 0x7f502c0cb520 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.190+0000 7f503217c700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c06c170 con 0x7f502c06cf30 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.190+0000 7f502b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f502c15cd30 0x7f502c0cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51412/0 (socket says 172.21.15.83:51412) 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.190+0000 7f502b7fe700 1 -- 172.21.15.83:0/1118315092 learned_addr learned my addr 172.21.15.83:0/1118315092 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502bfff700 1 --2- 172.21.15.83:0/1118315092 >> v2:172.21.15.90:3300/0 conn(0x7f502c06cf30 0x7f502c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316456989 0 0) 0x7f5020001110 con 0x7f502c15cd30 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50140035b0 con 0x7f502c15cd30 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038306961 0 0) 0x7f50140035b0 con 0x7f502c15cd30 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 >> v2:172.21.15.90:3300/0 conn(0x7f502c06cf30 msgr2=0x7f502c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 --2- 172.21.15.83:0/1118315092 >> v2:172.21.15.90:3300/0 conn(0x7f502c06cf30 0x7f502c0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 >> v1:172.21.15.90:6789/0 conn(0x7f502c0cb520 legacy=0x7f502c0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c06c450 con 0x7f502c15cd30 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f5020001800 con 0x7f502c15cd30 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 >> v1:172.21.15.83:6789/0 conn(0x7f502c15cd30 legacy=0x7f502c0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5014003a00 0x7f5014003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014004850 0x7f5014004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5014005200 0x7f501400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c06c170 con 0x7f5014003a00 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501400c080 con 0x7f5014004850 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501400c240 con 0x7f5014005200 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502b7fe700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5014003a00 0x7f5014003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502affd700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014004850 0x7f5014004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502bfff700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5014005200 0x7f501400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502b7fe700 1 -- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5014005200 msgr2=0x7f501400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502b7fe700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5014005200 0x7f501400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502b7fe700 1 -- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014004850 msgr2=0x7f5014004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502b7fe700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014004850 0x7f5014004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502b7fe700 1 -- 172.21.15.83:0/1118315092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140035b0 con 0x7f5014003a00 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.191+0000 7f502bfff700 1 -- 172.21.15.83:0/1118315092 reap_dead start 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f502b7fe700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5014003a00 0x7f5014003e10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f502000c240 tx=0x7f5014005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27093b5a020ae15b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020007db0 con 0x7f5014003a00 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000cd80 con 0x7f5014003a00 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f502a7fc700 1 -- 172.21.15.83:0/1118315092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020005ba0 con 0x7f5014003a00 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 -- 172.21.15.83:0/1118315092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5014003a00 msgr2=0x7f5014003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5014003a00 0x7f5014003e10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f502000c240 tx=0x7f5014005880 comp rx=0 tx=0).stop 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 -- 172.21.15.83:0/1118315092 shutdown_connections 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 --2- 172.21.15.83:0/1118315092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5014003a00 0x7f5014003e10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 -- 172.21.15.83:0/1118315092 >> 172.21.15.83:0/1118315092 conn(0x7f502c0cc690 msgr2=0x7f502c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 -- 172.21.15.83:0/1118315092 shutdown_connections 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.192+0000 7f503217c700 1 -- 172.21.15.83:0/1118315092 wait complete. 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.193+0000 7f503217c700 1 Processor -- start 2023-11-28T11:12:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 -- start start 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014003a00 0x7f502c14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f502c1e3e60 0x7f502c1e2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 0x7f502c1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1e2dc0 con 0x7f502c1e25d0 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1e30a0 con 0x7f5014003a00 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.194+0000 7f503217c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1e3380 con 0x7f502c1e3e60 2023-11-28T11:12:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 0x7f502c1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 0x7f502c1f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53568/0 (socket says 172.21.15.83:53568) 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 -- 172.21.15.83:0/982129483 learned_addr learned my addr 172.21.15.83:0/982129483 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502b7fe700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014003a00 0x7f502c14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502affd700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f502c1e3e60 0x7f502c1e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 -- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f502c1e3e60 msgr2=0x7f502c1e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f502c1e3e60 0x7f502c1e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 -- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014003a00 msgr2=0x7f502c14eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014003a00 0x7f502c14eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 -- 172.21.15.83:0/982129483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c06ca10 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f502bfff700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 0x7f502c1f2700 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f502c0c6c70 tx=0x7f50240114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502400e070 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024003c30 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c06c730 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024003e20 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.195+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c0cae10 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.196+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c0742b0 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.196+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5024011a10 con 0x7f502c1e25d0 2023-11-28T11:12:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.197+0000 7f5013fff700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50180682f0 0x7f501806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.197+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6088+0+0 (secure 0 0 0) 0x7f50240861f0 con 0x7f502c1e25d0 2023-11-28T11:12:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.197+0000 7f502b7fe700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50180682f0 0x7f501806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.197+0000 7f502b7fe700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50180682f0 0x7f501806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5020001ac0 tx=0x7f5020007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.201+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5024052950 con 0x7f502c1e25d0 2023-11-28T11:12:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:39.387+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f502c0aeba0 con 0x7f502c1e25d0 2023-11-28T11:12:40.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.054+0000 7f5013fff700 1 -- 172.21.15.83:0/982129483 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5024055f70 con 0x7f502c1e25d0 2023-11-28T11:12:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.058+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50180682f0 msgr2=0x7f501806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.058+0000 7f503217c700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50180682f0 0x7f501806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5020001ac0 tx=0x7f5020007000 comp rx=0 tx=0).stop 2023-11-28T11:12:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.058+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 msgr2=0x7f502c1f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.058+0000 7f503217c700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 0x7f502c1f2700 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f502c0c6c70 tx=0x7f50240114b0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.060+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 shutdown_connections 2023-11-28T11:12:40.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.060+0000 7f503217c700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f50180682f0 0x7f501806a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.060+0000 7f503217c700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f502c1e25d0 0x7f502c1f2700 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.060+0000 7f503217c700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5014003a00 0x7f502c14eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.060+0000 7f503217c700 1 --2- 172.21.15.83:0/982129483 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f502c1e3e60 0x7f502c1e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.060+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 >> 172.21.15.83:0/982129483 conn(0x7f502c0cc690 msgr2=0x7f502c0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:40.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.061+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 shutdown_connections 2023-11-28T11:12:40.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.062+0000 7f503217c700 1 -- 172.21.15.83:0/982129483 wait complete. 2023-11-28T11:12:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-28T11:12:40.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:12:40.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.202+0000 7f28c7074700 1 Processor -- start 2023-11-28T11:12:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.202+0000 7f28c7074700 1 -- start start 2023-11-28T11:12:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c7074700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28c0158bc0 0x7f28c00cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c7074700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c00ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c7074700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c0158f90 con 0x7f28c00cb680 2023-11-28T11:12:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c7074700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c015c840 con 0x7f28c0158bc0 2023-11-28T11:12:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c7074700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015cb20 con 0x7f28c0071a50 2023-11-28T11:12:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28c0158bc0 0x7f28c00cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28c0158bc0 0x7f28c00cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53572/0 (socket says 172.21.15.83:53572) 2023-11-28T11:12:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 -- 172.21.15.83:0/1866268562 learned_addr learned my addr 172.21.15.83:0/1866268562 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c5611700 1 --2- 172.21.15.83:0/1866268562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c00ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 -- 172.21.15.83:0/1866268562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 msgr2=0x7f28c00ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 --2- 172.21.15.83:0/1866268562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c00ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 -- 172.21.15.83:0/1866268562 >> v1:172.21.15.90:6789/0 conn(0x7f28c00cb680 legacy=0x7f28c0071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.203+0000 7f28c4e10700 1 -- 172.21.15.83:0/1866268562 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c015ce00 con 0x7f28c0158bc0 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c4e10700 1 --2- 172.21.15.83:0/1866268562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c0158bc0 0x7f28c00cdd70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f28b400bb90 tx=0x7f28b400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809cb273dc9eff5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28bf7fe700 1 -- 172.21.15.83:0/1866268562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400dc40 con 0x7f28c0158bc0 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28bf7fe700 1 -- 172.21.15.83:0/1866268562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400dda0 con 0x7f28c0158bc0 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28bf7fe700 1 -- 172.21.15.83:0/1866268562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4017630 con 0x7f28c0158bc0 2023-11-28T11:12:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 -- 172.21.15.83:0/1866268562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c0158bc0 msgr2=0x7f28c00cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 --2- 172.21.15.83:0/1866268562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c0158bc0 0x7f28c00cdd70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f28b400bb90 tx=0x7f28b400bea0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 -- 172.21.15.83:0/1866268562 shutdown_connections 2023-11-28T11:12:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 --2- 172.21.15.83:0/1866268562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c00ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 --2- 172.21.15.83:0/1866268562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c0158bc0 0x7f28c00cdd70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 -- 172.21.15.83:0/1866268562 >> 172.21.15.83:0/1866268562 conn(0x7f28c00cf300 msgr2=0x7f28c00cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 -- 172.21.15.83:0/1866268562 shutdown_connections 2023-11-28T11:12:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.204+0000 7f28c7074700 1 -- 172.21.15.83:0/1866268562 wait complete. 2023-11-28T11:12:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 Processor -- start 2023-11-28T11:12:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 -- start start 2023-11-28T11:12:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c01566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 0x7f28c0155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28c0158bc0 0x7f28c01432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00b2130 con 0x7f28c00cb680 2023-11-28T11:12:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0156260 con 0x7f28c0158bc0 2023-11-28T11:12:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.206+0000 7f28c7074700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00c5c50 con 0x7f28c0071a50 2023-11-28T11:12:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 0x7f28c0155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 0x7f28c0155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53580/0 (socket says 172.21.15.83:53580) 2023-11-28T11:12:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 -- 172.21.15.83:0/2105365439 learned_addr learned my addr 172.21.15.83:0/2105365439 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 -- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 msgr2=0x7f28c01566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28c4e10700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c01566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28c5611700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28c0158bc0 0x7f28c01432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c01566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 -- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28c0158bc0 msgr2=0x7f28c01432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28c0158bc0 0x7f28c01432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 -- 172.21.15.83:0/2105365439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b400b7f0 con 0x7f28c00cb680 2023-11-28T11:12:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bffff700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 0x7f28c0155410 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00c540 tx=0x7f28ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0052d0 con 0x7f28c00cb680 2023-11-28T11:12:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac005430 con 0x7f28c00cb680 2023-11-28T11:12:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0038c0 con 0x7f28c00cb680 2023-11-28T11:12:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c015d3c0 con 0x7f28c00cb680 2023-11-28T11:12:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.207+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c015d0e0 con 0x7f28c00cb680 2023-11-28T11:12:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.208+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c00ce2b0 con 0x7f28c00cb680 2023-11-28T11:12:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.208+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f28ac019030 con 0x7f28c00cb680 2023-11-28T11:12:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.209+0000 7f28bd7fa700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f28a40682f0 0x7f28a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.209+0000 7f28c4e10700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f28a40682f0 0x7f28a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.209+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f28ac084e30 con 0x7f28c00cb680 2023-11-28T11:12:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.209+0000 7f28c4e10700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f28a40682f0 0x7f28a406a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28b4019040 tx=0x7f28b4014000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.213+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f28ac0517f0 con 0x7f28c00cb680 2023-11-28T11:12:40.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:40.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.409+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f28c00742b0 con 0x7f28c00cb680 2023-11-28T11:12:40.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.413+0000 7f28bd7fa700 1 -- 172.21.15.83:0/2105365439 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f28ac054e10 con 0x7f28c00cb680 2023-11-28T11:12:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f28a40682f0 msgr2=0x7f28a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f28a40682f0 0x7f28a406a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28b4019040 tx=0x7f28b4014000 comp rx=0 tx=0).stop 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 msgr2=0x7f28c0155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 0x7f28c0155410 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00c540 tx=0x7f28ac00c900 comp rx=0 tx=0).stop 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 shutdown_connections 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28c0158bc0 0x7f28c01432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f28c0071a50 0x7f28c01566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f28a40682f0 0x7f28a406a7a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 --2- 172.21.15.83:0/2105365439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28c00cb680 0x7f28c0155410 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 >> 172.21.15.83:0/2105365439 conn(0x7f28c00cf300 msgr2=0x7f28c00cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 shutdown_connections 2023-11-28T11:12:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.415+0000 7f28c7074700 1 -- 172.21.15.83:0/2105365439 wait complete. 2023-11-28T11:12:40.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:12:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.547+0000 7f7b06347700 1 Processor -- start 2023-11-28T11:12:40.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.547+0000 7f7b06347700 1 -- start start 2023-11-28T11:12:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7b06347700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7b0015c9e0 0x7f7b0006ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7b06347700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b000b5220 0x7f7b000d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7b06347700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0006b1c0 con 0x7f7b000b5ee0 2023-11-28T11:12:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7b06347700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000cda10 con 0x7f7b0015c9e0 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7b06347700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000cdcf0 con 0x7f7b000b5220 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7aff7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7b000b5ee0 0x7f7b000b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51448/0 (socket says 172.21.15.83:51448) 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7aff7fe700 1 -- 172.21.15.83:0/3162715560 learned_addr learned my addr 172.21.15.83:0/3162715560 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afffff700 1 --2- 172.21.15.83:0/3162715560 >> v2:172.21.15.90:3300/0 conn(0x7f7b0015c9e0 0x7f7b0006ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7b048e4700 1 --2- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b000b5220 0x7f7b000d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afeffd700 1 -- 172.21.15.83:0/3162715560 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603789901 0 0) 0x7f7b0006b1c0 con 0x7f7b000b5ee0 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afeffd700 1 -- 172.21.15.83:0/3162715560 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae80035b0 con 0x7f7b000b5ee0 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afffff700 1 -- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b000b5220 msgr2=0x7f7b000d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afffff700 1 --2- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b000b5220 0x7f7b000d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afffff700 1 -- 172.21.15.83:0/3162715560 >> v1:172.21.15.83:6789/0 conn(0x7f7b000b5ee0 legacy=0x7f7b000b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afeffd700 1 -- 172.21.15.83:0/3162715560 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828679454 0 0) 0x7f7ae80035b0 con 0x7f7b000b5ee0 2023-11-28T11:12:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.548+0000 7f7afffff700 1 -- 172.21.15.83:0/3162715560 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000d0c60 con 0x7f7b0015c9e0 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7afffff700 1 --2- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0015c9e0 0x7f7b0006ac80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009840 tx=0x7f7af4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955771f86a9b8010 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7afeffd700 1 -- 172.21.15.83:0/3162715560 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0001670 con 0x7f7b0015c9e0 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7afeffd700 1 -- 172.21.15.83:0/3162715560 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4004030 con 0x7f7b0015c9e0 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7afeffd700 1 -- 172.21.15.83:0/3162715560 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40043c0 con 0x7f7b0015c9e0 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 -- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0015c9e0 msgr2=0x7f7b0006ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 --2- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0015c9e0 0x7f7b0006ac80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009840 tx=0x7f7af4009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 -- 172.21.15.83:0/3162715560 shutdown_connections 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 --2- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b000b5220 0x7f7b000d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 --2- 172.21.15.83:0/3162715560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0015c9e0 0x7f7b0006ac80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 -- 172.21.15.83:0/3162715560 >> 172.21.15.83:0/3162715560 conn(0x7f7b000cfb10 msgr2=0x7f7b000cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 -- 172.21.15.83:0/3162715560 shutdown_connections 2023-11-28T11:12:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.549+0000 7f7b06347700 1 -- 172.21.15.83:0/3162715560 wait complete. 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.550+0000 7f7b06347700 1 Processor -- start 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 -- start start 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 0x7f7b00143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b000b5ee0 0x7f7b000c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b0015c9e0 0x7f7b00145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00144450 con 0x7f7b000b5220 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000c9fb0 con 0x7f7b000b5ee0 2023-11-28T11:12:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7b06347700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00145270 con 0x7f7b0015c9e0 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7afffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 0x7f7b00143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7afffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 0x7f7b00143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53594/0 (socket says 172.21.15.83:53594) 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.551+0000 7f7afffff700 1 -- 172.21.15.83:0/3035224184 learned_addr learned my addr 172.21.15.83:0/3035224184 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7aff7fe700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b000b5ee0 0x7f7b000c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7b048e4700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b0015c9e0 0x7f7b00145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afffff700 1 -- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b0015c9e0 msgr2=0x7f7b00145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afffff700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b0015c9e0 0x7f7b00145700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afffff700 1 -- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b000b5ee0 msgr2=0x7f7b000c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afffff700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b000b5ee0 0x7f7b000c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afffff700 1 -- 172.21.15.83:0/3035224184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40094a0 con 0x7f7b000b5220 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7aff7fe700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b000b5ee0 0x7f7b000c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afffff700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 0x7f7b00143f10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009790 tx=0x7f7af4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003b60 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af401f070 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b000d1220 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011630 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.552+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b000d0f40 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.553+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b000cd6e0 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.553+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7af402a020 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.554+0000 7f7afcff9700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7aec0682f0 0x7f7aec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.554+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7af4086ed0 con 0x7f7b000b5220 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.554+0000 7f7aff7fe700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7aec0682f0 0x7f7aec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.554+0000 7f7aff7fe700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7aec0682f0 0x7f7aec06a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7af0001eb0 tx=0x7f7af0001d80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:40.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.558+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7af40538f0 con 0x7f7b000b5220 2023-11-28T11:12:40.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:40.815+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f7b00074320 con 0x7f7b000b5220 2023-11-28T11:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.055+0000 7f7afcff9700 1 -- 172.21.15.83:0/3035224184 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v211) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7af4056f10 con 0x7f7b000b5220 2023-11-28T11:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7aec0682f0 msgr2=0x7f7aec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7aec0682f0 0x7f7aec06a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7af0001eb0 tx=0x7f7af0001d80 comp rx=0 tx=0).stop 2023-11-28T11:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 msgr2=0x7f7b00143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 0x7f7b00143f10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009790 tx=0x7f7af4004750 comp rx=0 tx=0).stop 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 shutdown_connections 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b0015c9e0 0x7f7b00145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f7aec0682f0 0x7f7aec06a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b000b5220 0x7f7b00143f10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 --2- 172.21.15.83:0/3035224184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b000b5ee0 0x7f7b000c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.057+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 >> 172.21.15.83:0/3035224184 conn(0x7f7b000cfb10 msgr2=0x7f7b000cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.058+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 shutdown_connections 2023-11-28T11:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:41.058+0000 7f7b06347700 1 -- 172.21.15.83:0/3035224184 wait complete. 2023-11-28T11:12:41.072 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T11:12:41.072 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:41.072 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:41.073 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:Ran 3 tests in 203.268s 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:12:41.074 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:12:41.075 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:12:41.075 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:41.075 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:41.178 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=363b88a520f44698996166f1fa938c05 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:41.180 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T11:12:41.188 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T11:12:41.199 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T11:12:41.199 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:12:41.199 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:41.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:12:41.221 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-28T11:12:41.221 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:41.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20538 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:12:41.279 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:12:41.279 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:41.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:41.300 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-11-28T11:12:41.300 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:41.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284208k,nr_inodes=4071052,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=746 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:41.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.361 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:12:41.378 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:12:41.378 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:12:41.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:12:41.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:12:41.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-28T11:12:41.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.642 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-28T11:12:41.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-28T11:12:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2023-11-28T11:12:41.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:41.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.732 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2023-11-28T11:12:41.753 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T11:12:41.753 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:12:41.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.754 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:12:41.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.833 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:12:41.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:41.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2023-11-28T11:12:42.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:42.015 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2023-11-28T11:12:42.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:42.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2023-11-28T11:12:42.101 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2023-11-28T11:12:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:42.103 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T11:12:42.112 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:12:42.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.234+0000 7fc04b651700 1 Processor -- start 2023-11-28T11:12:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.234+0000 7fc04b651700 1 -- start start 2023-11-28T11:12:42.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc044071710 0x7fc0440cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc0440c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0440cac90 con 0x7fc0440c9de0 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0440ca1f0 con 0x7fc044071710 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440ca3f0 con 0x7fc044071ae0 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc049bee700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc0440c9de0 0x7fc0441586b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51462/0 (socket says 172.21.15.83:51462) 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc049bee700 1 -- 172.21.15.83:0/2901336584 learned_addr learned my addr 172.21.15.83:0/2901336584 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc048bec700 1 --2- 172.21.15.83:0/2901336584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc0440c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc0493ed700 1 --2- 172.21.15.83:0/2901336584 >> v2:172.21.15.90:3300/0 conn(0x7fc044071710 0x7fc0440cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc03bfff700 1 -- 172.21.15.83:0/2901336584 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461462000 0 0) 0x7fc0440cac90 con 0x7fc0440c9de0 2023-11-28T11:12:42.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc03bfff700 1 -- 172.21.15.83:0/2901336584 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0300035b0 con 0x7fc0440c9de0 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc03bfff700 1 -- 172.21.15.83:0/2901336584 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1166005854 0 0) 0x7fc0300035b0 con 0x7fc0440c9de0 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc048bec700 1 -- 172.21.15.83:0/2901336584 >> v2:172.21.15.90:3300/0 conn(0x7fc044071710 msgr2=0x7fc0440cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc048bec700 1 --2- 172.21.15.83:0/2901336584 >> v2:172.21.15.90:3300/0 conn(0x7fc044071710 0x7fc0440cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc048bec700 1 -- 172.21.15.83:0/2901336584 >> v1:172.21.15.83:6789/0 conn(0x7fc0440c9de0 legacy=0x7fc0441586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc048bec700 1 -- 172.21.15.83:0/2901336584 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440ca6d0 con 0x7fc044071ae0 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc048bec700 1 --2- 172.21.15.83:0/2901336584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc0440c98a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc040009840 tx=0x7fc040009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5864122b423ae457 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc03bfff700 1 -- 172.21.15.83:0/2901336584 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0019f0 con 0x7fc044071ae0 2023-11-28T11:12:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc03bfff700 1 -- 172.21.15.83:0/2901336584 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040004030 con 0x7fc044071ae0 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc03bfff700 1 -- 172.21.15.83:0/2901336584 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0400043c0 con 0x7fc044071ae0 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- 172.21.15.83:0/2901336584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 msgr2=0x7fc0440c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 --2- 172.21.15.83:0/2901336584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc0440c98a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc040009840 tx=0x7fc040009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- 172.21.15.83:0/2901336584 shutdown_connections 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 --2- 172.21.15.83:0/2901336584 >> v2:172.21.15.90:3300/0 conn(0x7fc044071710 0x7fc0440cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 --2- 172.21.15.83:0/2901336584 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc0440c98a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- 172.21.15.83:0/2901336584 >> 172.21.15.83:0/2901336584 conn(0x7fc0440cec90 msgr2=0x7fc0440cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- 172.21.15.83:0/2901336584 shutdown_connections 2023-11-28T11:12:42.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.236+0000 7fc04b651700 1 -- 172.21.15.83:0/2901336584 wait complete. 2023-11-28T11:12:42.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 Processor -- start 2023-11-28T11:12:42.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 -- start start 2023-11-28T11:12:42.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc044071710 0x7fc044146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc044157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 0x7fc0440c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c6e40 con 0x7fc0440c9de0 2023-11-28T11:12:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c3ee0 con 0x7fc044071710 2023-11-28T11:12:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc04b651700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c3b90 con 0x7fc044071ae0 2023-11-28T11:12:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.238+0000 7fc049bee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 0x7fc0440c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 0x7fc0440c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53606/0 (socket says 172.21.15.83:53606) 2023-11-28T11:12:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 -- 172.21.15.83:0/2180720795 learned_addr learned my addr 172.21.15.83:0/2180720795 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc0493ed700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc044071710 0x7fc044146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc048bec700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc044157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 -- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 msgr2=0x7fc044157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc044157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 -- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc044071710 msgr2=0x7fc044146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc044071710 0x7fc044146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 -- 172.21.15.83:0/2180720795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400094a0 con 0x7fc0440c9de0 2023-11-28T11:12:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc0493ed700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc044071710 0x7fc044146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc049bee700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 0x7fc0440c2740 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00ccd0 tx=0x7fc03c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00ada0 con 0x7fc0440c9de0 2023-11-28T11:12:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c0070d0 con 0x7fc0440c9de0 2023-11-28T11:12:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04406dae0 con 0x7fc0440c9de0 2023-11-28T11:12:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c007480 con 0x7fc0440c9de0 2023-11-28T11:12:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.239+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0440ca9b0 con 0x7fc0440c9de0 2023-11-28T11:12:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.240+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc03c01a030 con 0x7fc0440c9de0 2023-11-28T11:12:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.240+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440c24a0 con 0x7fc0440c9de0 2023-11-28T11:12:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.241+0000 7fc039ffb700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0280682f0 0x7fc02806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.241+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc03c0858e0 con 0x7fc0440c9de0 2023-11-28T11:12:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.241+0000 7fc0493ed700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0280682f0 0x7fc02806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.241+0000 7fc0493ed700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0280682f0 0x7fc02806a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc034009ac0 tx=0x7fc034008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.246+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc03c052300 con 0x7fc0440c9de0 2023-11-28T11:12:42.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.422+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc044074320 con 0x7fc0280682f0 2023-11-28T11:12:42.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.425+0000 7fc039ffb700 1 -- 172.21.15.83:0/2180720795 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73137 (secure 0 0 0) 0x7fc044074320 con 0x7fc0280682f0 2023-11-28T11:12:42.427 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0280682f0 msgr2=0x7fc02806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0280682f0 0x7fc02806a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc034009ac0 tx=0x7fc034008040 comp rx=0 tx=0).stop 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 msgr2=0x7fc0440c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 0x7fc0440c2740 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00ccd0 tx=0x7fc03c00a340 comp rx=0 tx=0).stop 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 shutdown_connections 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fc0280682f0 0x7fc02806a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0440c9de0 0x7fc0440c2740 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc044071710 0x7fc044146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 --2- 172.21.15.83:0/2180720795 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc044071ae0 0x7fc044157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 >> 172.21.15.83:0/2180720795 conn(0x7fc0440cec90 msgr2=0x7fc04406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 shutdown_connections 2023-11-28T11:12:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.427+0000 7fc04b651700 1 -- 172.21.15.83:0/2180720795 wait complete. 2023-11-28T11:12:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:12:42.440 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-11-28T11:12:41.624414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":80688,"kb_used_data":7472,"kb_used_omap":0,"kb_used_meta":73152,"kb_avail":754894032,"statfs":{"total":773094113280,"available":773011488768,"internally_reserved":0,"allocated":7651328,"data_stored":4219578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74907648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001359"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.241155+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:12:26.241155+0000","last_peered":"2023-11-28T11:12:26.241155+0000","last_clean":"2023-11-28T11:12:26.241155+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:12:26.241155+0000","last_undegraded":"2023-11-28T11:12:26.241155+0000","last_fullsized":"2023-11-28T11:12:26.241155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:27.108867+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:12:27.108867+0000","last_peered":"2023-11-28T11:12:27.108867+0000","last_clean":"2023-11-28T11:12:27.108867+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:12:27.108867+0000","last_undegraded":"2023-11-28T11:12:27.108867+0000","last_fullsized":"2023-11-28T11:12:27.108867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:25.960404+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:12:25.960404+0000","last_peered":"2023-11-28T11:12:25.960404+0000","last_clean":"2023-11-28T11:12:25.960404+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:12:25.960404+0000","last_undegraded":"2023-11-28T11:12:25.960404+0000","last_fullsized":"2023-11-28T11:12:25.960404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":201,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:33.846534+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:12:33.846534+0000","last_peered":"2023-11-28T11:12:33.846534+0000","last_clean":"2023-11-28T11:12:33.846534+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:12:33.846534+0000","last_undegraded":"2023-11-28T11:12:33.846534+0000","last_fullsized":"2023-11-28T11:12:33.846534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:27.109239+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:12:27.109239+0000","last_peered":"2023-11-28T11:12:27.109239+0000","last_clean":"2023-11-28T11:12:27.109239+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:12:27.109239+0000","last_undegraded":"2023-11-28T11:12:27.109239+0000","last_fullsized":"2023-11-28T11:12:27.109239+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:33.846495+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:12:33.846495+0000","last_peered":"2023-11-28T11:12:33.846495+0000","last_clean":"2023-11-28T11:12:33.846495+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:12:33.846495+0000","last_undegraded":"2023-11-28T11:12:33.846495+0000","last_fullsized":"2023-11-28T11:12:33.846495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:25.962068+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:12:25.962068+0000","last_peered":"2023-11-28T11:12:25.962068+0000","last_clean":"2023-11-28T11:12:25.962068+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:12:25.962068+0000","last_undegraded":"2023-11-28T11:12:25.962068+0000","last_fullsized":"2023-11-28T11:12:25.962068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.379752+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:12:26.379752+0000","last_peered":"2023-11-28T11:12:26.379752+0000","last_clean":"2023-11-28T11:12:26.379752+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:12:26.379752+0000","last_undegraded":"2023-11-28T11:12:26.379752+0000","last_fullsized":"2023-11-28T11:12:26.379752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.379975+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:12:26.379975+0000","last_peered":"2023-11-28T11:12:26.379975+0000","last_clean":"2023-11-28T11:12:26.379975+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:12:26.379975+0000","last_undegraded":"2023-11-28T11:12:26.379975+0000","last_fullsized":"2023-11-28T11:12:26.379975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10912,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360928,"statfs":{"total":96636764160,"available":96625590272,"internally_reserved":0,"allocated":1400832,"data_stored":968021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.746,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.46700000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54200000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.48599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.442},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":375784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70199999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":1.544,"15min":1.544},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66000000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.499},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":1.665,"15min":1.665},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.70599999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69599999999999995,"5min":1.762,"15min":1.762},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.39800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.002,"15min":1.002},"last":0.438},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10868,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360972,"statfs":{"total":96636764160,"available":96625635328,"internally_reserved":0,"allocated":1421312,"data_stored":979809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.247,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54900000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"min":{"1min":0.25800000000000001,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.251,"5min":0.187,"15min":0.187},"max":{"1min":0.54400000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39800000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63800000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.627,"5min":1.363,"15min":1.363},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.622}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90600000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":1.48,"15min":1.48},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.498},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.42299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":794624,"data_stored":375692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.222},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.33300000000000002},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.248}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55600000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53000000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.312},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57499999999999996,"5min":0.871,"15min":0.871},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.379}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80100000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68600000000000005,"15min":0.754},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.877,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.434},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.45400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9768,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362072,"statfs":{"total":96636764160,"available":96626761728,"internally_reserved":0,"allocated":819200,"data_stored":388841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33300000000000002},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.313}]},{"osd":1,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32700000000000001},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.247},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.628,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.51000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71699999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79100000000000004,"5min":1.024,"15min":1.024},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80500000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.76600000000000001,"5min":1.05,"15min":1.05},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9884,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361956,"statfs":{"total":96636764160,"available":96626642944,"internally_reserved":0,"allocated":806912,"data_stored":377165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46800000000000003,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.222},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.48999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59399999999999997,"5min":1.054,"15min":1.054},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56299999999999994,"5min":1.147,"15min":1.147},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55700000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.49299999999999999,"15min":0.59099999999999997},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":5.133,"15min":5.133},"last":0.34000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.499},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.67500000000000004,"5min":0.877,"15min":0.877},"last":0.628}]},{"osd":5,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":1.194,"15min":1.194},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":1.077,"15min":1.077},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":802816,"data_stored":377123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.505}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.65800000000000003,"5min":0.753,"15min":0.753},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.54500000000000004,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72199999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":377143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.73799999999999999,"15min":0.83599999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.76700000000000002,"15min":0.873},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69399999999999995,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64400000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.70899999999999996,"15min":0.78900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":6.601,"15min":6.601},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71399999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.45100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.83499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:42.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.563+0000 7f1b99510700 1 Processor -- start 2023-11-28T11:12:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.563+0000 7f1b99510700 1 -- start start 2023-11-28T11:12:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b99510700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 0x7f1b940d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b99510700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b940cda10 con 0x7f1b9415c9e0 2023-11-28T11:12:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b99510700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b940cdcf0 con 0x7f1b940b5ee0 2023-11-28T11:12:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b99510700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940d0c60 con 0x7f1b940b5220 2023-11-28T11:12:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b92ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1b9415c9e0 0x7f1b9406ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51472/0 (socket says 172.21.15.83:51472) 2023-11-28T11:12:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b92ffd700 1 -- 172.21.15.83:0/2593072424 learned_addr learned my addr 172.21.15.83:0/2593072424 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b937fe700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 0x7f1b940d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428514812 0 0) 0x7f1b940cda10 con 0x7f1b9415c9e0 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.564+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b7c0035b0 con 0x7f1b9415c9e0 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795753555 0 0) 0x7f1b940cdcf0 con 0x7f1b940b5ee0 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b940cda10 con 0x7f1b940b5ee0 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860516081 0 0) 0x7f1b7c0035b0 con 0x7f1b9415c9e0 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 msgr2=0x7f1b940d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 0x7f1b940d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 >> v1:172.21.15.90:6789/0 conn(0x7f1b940b5ee0 legacy=0x7f1b940b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940d0f40 con 0x7f1b9415c9e0 2023-11-28T11:12:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7f1b88001670 con 0x7f1b9415c9e0 2023-11-28T11:12:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 >> v1:172.21.15.83:6789/0 conn(0x7f1b9415c9e0 legacy=0x7f1b9406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 0x7f1b7c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c004170 0x7f1b7c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7c004d90 0x7f1b7c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940d0c60 con 0x7f1b7c004d90 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00bb20 con 0x7f1b7c004170 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00bce0 con 0x7f1b940b5220 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7c004d90 0x7f1b7c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b92ffd700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 0x7f1b7c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b927fc700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c004170 0x7f1b7c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 -- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 msgr2=0x7f1b7c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940b5220 0x7f1b7c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 -- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c004170 msgr2=0x7f1b7c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c004170 0x7f1b7c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 -- 172.21.15.83:0/2593072424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0035b0 con 0x7f1b7c004d90 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b937fe700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7c004d90 0x7f1b7c005200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009fc0 tx=0x7f1b7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c844d4659c98b3c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00ec30 con 0x7f1b7c004d90 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c00f040 con 0x7f1b7c004d90 2023-11-28T11:12:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.565+0000 7f1b91ffb700 1 -- 172.21.15.83:0/2593072424 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c014480 con 0x7f1b7c004d90 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b99510700 1 -- 172.21.15.83:0/2593072424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7c004d90 msgr2=0x7f1b7c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b99510700 1 --2- 172.21.15.83:0/2593072424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7c004d90 0x7f1b7c005200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009fc0 tx=0x7f1b7c005880 comp rx=0 tx=0).stop 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b937fe700 1 -- 172.21.15.83:0/2593072424 reap_dead start 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b99510700 1 -- 172.21.15.83:0/2593072424 shutdown_connections 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b99510700 1 -- 172.21.15.83:0/2593072424 >> 172.21.15.83:0/2593072424 conn(0x7f1b940cfb10 msgr2=0x7f1b940cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b99510700 1 -- 172.21.15.83:0/2593072424 shutdown_connections 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.566+0000 7f1b99510700 1 -- 172.21.15.83:0/2593072424 wait complete. 2023-11-28T11:12:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.567+0000 7f1b99510700 1 Processor -- start 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.567+0000 7f1b99510700 1 -- start start 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b99510700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b94156780 0x7f1b941f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b99510700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b94156b50 0x7f1b9415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b99510700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 0x7f1b940d3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b99510700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b941f2a80 con 0x7f1b94156ff0 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b99510700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940a9980 con 0x7f1b94156780 2023-11-28T11:12:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b99510700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9415f600 con 0x7f1b94156b50 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 0x7f1b940d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 0x7f1b940d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53628/0 (socket says 172.21.15.83:53628) 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 -- 172.21.15.83:0/3936555069 learned_addr learned my addr 172.21.15.83:0/3936555069 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b92ffd700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b94156780 0x7f1b941f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b927fc700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b94156b50 0x7f1b9415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 -- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b94156b50 msgr2=0x7f1b9415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b94156b50 0x7f1b9415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 -- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b94156780 msgr2=0x7f1b941f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b94156780 0x7f1b941f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b937fe700 1 -- 172.21.15.83:0/3936555069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940cd6e0 con 0x7f1b94156ff0 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.568+0000 7f1b92ffd700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b94156780 0x7f1b941f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.569+0000 7f1b937fe700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 0x7f1b940d3090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c016000 tx=0x7f1b8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.569+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c007000 con 0x7f1b94156ff0 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.569+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c022070 con 0x7f1b94156ff0 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.569+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b940d1220 con 0x7f1b94156ff0 2023-11-28T11:12:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.569+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c029480 con 0x7f1b94156ff0 2023-11-28T11:12:42.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.569+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b940cf670 con 0x7f1b94156ff0 2023-11-28T11:12:42.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.570+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b94074320 con 0x7f1b94156ff0 2023-11-28T11:12:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.570+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b8c033430 con 0x7f1b94156ff0 2023-11-28T11:12:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.571+0000 7f1b7b7fe700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b80068320 0x7f1b8006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.571+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1b8c024070 con 0x7f1b94156ff0 2023-11-28T11:12:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.571+0000 7f1b92ffd700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b80068320 0x7f1b8006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.571+0000 7f1b92ffd700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b80068320 0x7f1b8006a7d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1b88000c00 tx=0x7f1b88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.574+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b8c054bb0 con 0x7f1b94156ff0 2023-11-28T11:12:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.753+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b941590e0 con 0x7f1b80068320 2023-11-28T11:12:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.755+0000 7f1b7b7fe700 1 -- 172.21.15.83:0/3936555069 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73137 (secure 0 0 0) 0x7f1b941590e0 con 0x7f1b80068320 2023-11-28T11:12:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b80068320 msgr2=0x7f1b8006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b80068320 0x7f1b8006a7d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1b88000c00 tx=0x7f1b88009040 comp rx=0 tx=0).stop 2023-11-28T11:12:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 msgr2=0x7f1b940d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 0x7f1b940d3090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c016000 tx=0x7f1b8c009750 comp rx=0 tx=0).stop 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 shutdown_connections 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f1b80068320 0x7f1b8006a7d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94156ff0 0x7f1b940d3090 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b94156780 0x7f1b941f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 --2- 172.21.15.83:0/3936555069 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b94156b50 0x7f1b9415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 >> 172.21.15.83:0/3936555069 conn(0x7f1b940cfb10 msgr2=0x7f1b9415bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 shutdown_connections 2023-11-28T11:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.758+0000 7f1b99510700 1 -- 172.21.15.83:0/3936555069 wait complete. 2023-11-28T11:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:12:42.771 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-11-28T11:12:41.624414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":80688,"kb_used_data":7472,"kb_used_omap":0,"kb_used_meta":73152,"kb_avail":754894032,"statfs":{"total":773094113280,"available":773011488768,"internally_reserved":0,"allocated":7651328,"data_stored":4219578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74907648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001359"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.241155+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:12:26.241155+0000","last_peered":"2023-11-28T11:12:26.241155+0000","last_clean":"2023-11-28T11:12:26.241155+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:12:26.241155+0000","last_undegraded":"2023-11-28T11:12:26.241155+0000","last_fullsized":"2023-11-28T11:12:26.241155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:27.108867+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:12:27.108867+0000","last_peered":"2023-11-28T11:12:27.108867+0000","last_clean":"2023-11-28T11:12:27.108867+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:12:27.108867+0000","last_undegraded":"2023-11-28T11:12:27.108867+0000","last_fullsized":"2023-11-28T11:12:27.108867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:25.960404+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:12:25.960404+0000","last_peered":"2023-11-28T11:12:25.960404+0000","last_clean":"2023-11-28T11:12:25.960404+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:12:25.960404+0000","last_undegraded":"2023-11-28T11:12:25.960404+0000","last_fullsized":"2023-11-28T11:12:25.960404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":201,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:33.846534+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:12:33.846534+0000","last_peered":"2023-11-28T11:12:33.846534+0000","last_clean":"2023-11-28T11:12:33.846534+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:12:33.846534+0000","last_undegraded":"2023-11-28T11:12:33.846534+0000","last_fullsized":"2023-11-28T11:12:33.846534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:27.109239+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:12:27.109239+0000","last_peered":"2023-11-28T11:12:27.109239+0000","last_clean":"2023-11-28T11:12:27.109239+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:12:27.109239+0000","last_undegraded":"2023-11-28T11:12:27.109239+0000","last_fullsized":"2023-11-28T11:12:27.109239+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:33.846495+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:12:33.846495+0000","last_peered":"2023-11-28T11:12:33.846495+0000","last_clean":"2023-11-28T11:12:33.846495+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:12:33.846495+0000","last_undegraded":"2023-11-28T11:12:33.846495+0000","last_fullsized":"2023-11-28T11:12:33.846495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:25.962068+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:12:25.962068+0000","last_peered":"2023-11-28T11:12:25.962068+0000","last_clean":"2023-11-28T11:12:25.962068+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:12:25.962068+0000","last_undegraded":"2023-11-28T11:12:25.962068+0000","last_fullsized":"2023-11-28T11:12:25.962068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.379752+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:12:26.379752+0000","last_peered":"2023-11-28T11:12:26.379752+0000","last_clean":"2023-11-28T11:12:26.379752+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:12:26.379752+0000","last_undegraded":"2023-11-28T11:12:26.379752+0000","last_fullsized":"2023-11-28T11:12:26.379752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.379975+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:12:26.379975+0000","last_peered":"2023-11-28T11:12:26.379975+0000","last_clean":"2023-11-28T11:12:26.379975+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:12:26.379975+0000","last_undegraded":"2023-11-28T11:12:26.379975+0000","last_fullsized":"2023-11-28T11:12:26.379975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10912,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360928,"statfs":{"total":96636764160,"available":96625590272,"internally_reserved":0,"allocated":1400832,"data_stored":968021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.746,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.46700000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54200000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.48599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.442},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":375784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70199999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":1.544,"15min":1.544},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66000000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.499},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":1.665,"15min":1.665},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.70599999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69599999999999995,"5min":1.762,"15min":1.762},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.39800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.002,"15min":1.002},"last":0.438},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10868,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360972,"statfs":{"total":96636764160,"available":96625635328,"internally_reserved":0,"allocated":1421312,"data_stored":979809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.247,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54900000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"min":{"1min":0.25800000000000001,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.251,"5min":0.187,"15min":0.187},"max":{"1min":0.54400000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39800000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63800000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.627,"5min":1.363,"15min":1.363},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.622}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90600000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":1.48,"15min":1.48},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.498},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.42299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":794624,"data_stored":375692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.222},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.33300000000000002},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.248}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55600000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53000000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.312},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57499999999999996,"5min":0.871,"15min":0.871},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.379}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80100000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68600000000000005,"15min":0.754},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.877,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.434},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.45400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9768,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362072,"statfs":{"total":96636764160,"available":96626761728,"internally_reserved":0,"allocated":819200,"data_stored":388841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33300000000000002},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.313}]},{"osd":1,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32700000000000001},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.247},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.628,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.51000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71699999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79100000000000004,"5min":1.024,"15min":1.024},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80500000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.76600000000000001,"5min":1.05,"15min":1.05},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9884,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361956,"statfs":{"total":96636764160,"available":96626642944,"internally_reserved":0,"allocated":806912,"data_stored":377165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46800000000000003,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.222},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.48999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59399999999999997,"5min":1.054,"15min":1.054},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56299999999999994,"5min":1.147,"15min":1.147},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55700000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.49299999999999999,"15min":0.59099999999999997},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":5.133,"15min":5.133},"last":0.34000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.499},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.67500000000000004,"5min":0.877,"15min":0.877},"last":0.628}]},{"osd":5,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":1.194,"15min":1.194},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":1.077,"15min":1.077},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":802816,"data_stored":377123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.505}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.65800000000000003,"5min":0.753,"15min":0.753},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.54500000000000004,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72199999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":377143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.73799999999999999,"15min":0.83599999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.76700000000000002,"15min":0.873},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69399999999999995,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64400000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.70899999999999996,"15min":0.78900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":6.601,"15min":6.601},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71399999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.45100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.83499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:42.774 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:12:42.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.893+0000 7fb11084c700 1 Processor -- start 2023-11-28T11:12:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.893+0000 7fb11084c700 1 -- start start 2023-11-28T11:12:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.894+0000 7fb11084c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 0x7fb10c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.894+0000 7fb11084c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0cbfb0 con 0x7fb10c15d0e0 2023-11-28T11:12:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.894+0000 7fb11084c700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0cb710 con 0x7fb10c15cd10 2023-11-28T11:12:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.894+0000 7fb11084c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0cb9f0 con 0x7fb10c0b1fb0 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb10c15d0e0 0x7fb10c06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51492/0 (socket says 172.21.15.83:51492) 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 -- 172.21.15.83:0/1035139316 learned_addr learned my addr 172.21.15.83:0/1035139316 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10ad9d700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 0x7fb10c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903307804 0 0) 0x7fb10c0cbfb0 con 0x7fb10c15d0e0 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f40035b0 con 0x7fb10c15d0e0 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 145416490 0 0) 0x7fb0f40035b0 con 0x7fb10c15d0e0 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 msgr2=0x7fb10c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 0x7fb10c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 >> v1:172.21.15.90:6789/0 conn(0x7fb10c15cd10 legacy=0x7fb10c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0cbcd0 con 0x7fb10c15d0e0 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fb0fc001670 con 0x7fb10c15d0e0 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 >> v1:172.21.15.83:6789/0 conn(0x7fb10c15d0e0 legacy=0x7fb10c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 0x7fb0f4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb0f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0cb9f0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400baf0 con 0x7fb0f4004d90 2023-11-28T11:12:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400bcb0 con 0x7fb10c0b1fb0 2023-11-28T11:12:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb0f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10a59c700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 0x7fb0f4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb10ad9d700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 -- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 msgr2=0x7fb0f4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c0b1fb0 0x7fb0f4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 -- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0f4004d90 msgr2=0x7fb0f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.895+0000 7fb109d9b700 1 -- 172.21.15.83:0/1035139316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40035b0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb109d9b700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb0f4004850 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c210 tx=0x7fb0f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d22f803d10829f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc007c10 con 0x7fb0f4003fb0 2023-11-28T11:12:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00cdb0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb10959a700 1 -- 172.21.15.83:0/1035139316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc005ba0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb11084c700 1 -- 172.21.15.83:0/1035139316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 msgr2=0x7fb0f4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb11084c700 1 --2- 172.21.15.83:0/1035139316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb0f4004850 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c210 tx=0x7fb0f4005880 comp rx=0 tx=0).stop 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb10ad9d700 1 -- 172.21.15.83:0/1035139316 reap_dead start 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb11084c700 1 -- 172.21.15.83:0/1035139316 shutdown_connections 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb11084c700 1 -- 172.21.15.83:0/1035139316 >> 172.21.15.83:0/1035139316 conn(0x7fb10c06e9a0 msgr2=0x7fb10c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb11084c700 1 -- 172.21.15.83:0/1035139316 shutdown_connections 2023-11-28T11:12:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.896+0000 7fb11084c700 1 -- 172.21.15.83:0/1035139316 wait complete. 2023-11-28T11:12:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 Processor -- start 2023-11-28T11:12:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 -- start start 2023-11-28T11:12:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb10c15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c1efce0 0x7fb10c1f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb10c1f0150 0x7fb10c0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1f39e0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b5b00 con 0x7fb10c1f0150 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb11084c700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1f3420 con 0x7fb10c1efce0 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.898+0000 7fb10a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb10c15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb10c15ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53644/0 (socket says 172.21.15.83:53644) 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 -- 172.21.15.83:0/3446495874 learned_addr learned my addr 172.21.15.83:0/3446495874 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10ad9d700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb10c1f0150 0x7fb10c0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb109d9b700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c1efce0 0x7fb10c1f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 -- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c1efce0 msgr2=0x7fb10c1f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c1efce0 0x7fb10c1f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 -- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb10c1f0150 msgr2=0x7fb10c0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb10c1f0150 0x7fb10c0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 -- 172.21.15.83:0/3446495874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c06b660 con 0x7fb0f4003fb0 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10ad9d700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb10c1f0150 0x7fb10c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb10a59c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb10c15ee80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb10000f7f0 tx=0x7fb10000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000052e0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb100005440 con 0x7fb0f4003fb0 2023-11-28T11:12:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c06b9b0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000038c0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.899+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c0cd9f0 con 0x7fb0f4003fb0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.900+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c074320 con 0x7fb0f4003fb0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.900+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb100003a20 con 0x7fb0f4003fb0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.901+0000 7fb0faffd700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb0ec0682f0 0x7fb0ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.901+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb100085a90 con 0x7fb0f4003fb0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.901+0000 7fb109d9b700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb0ec0682f0 0x7fb0ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:42.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.901+0000 7fb109d9b700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb0ec0682f0 0x7fb0ec06a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0c9ef0 tx=0x7fb0fc007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:42.905+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1000524b0 con 0x7fb0f4003fb0 2023-11-28T11:12:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.087+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb10c0b27b0 con 0x7fb0ec0682f0 2023-11-28T11:12:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.090+0000 7fb0faffd700 1 -- 172.21.15.83:0/3446495874 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73137 (secure 0 0 0) 0x7fb10c0b27b0 con 0x7fb0ec0682f0 2023-11-28T11:12:43.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb0ec0682f0 msgr2=0x7fb0ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb0ec0682f0 0x7fb0ec06a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0c9ef0 tx=0x7fb0fc007000 comp rx=0 tx=0).stop 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 msgr2=0x7fb10c15ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb10c15ee80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb10000f7f0 tx=0x7fb10000fbb0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 shutdown_connections 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb10c1f0150 0x7fb10c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb0ec0682f0 0x7fb0ec06a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4003fb0 0x7fb10c15ee80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 --2- 172.21.15.83:0/3446495874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb10c1efce0 0x7fb10c1f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 >> 172.21.15.83:0/3446495874 conn(0x7fb10c06e9a0 msgr2=0x7fb10c06bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 shutdown_connections 2023-11-28T11:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.092+0000 7fb11084c700 1 -- 172.21.15.83:0/3446495874 wait complete. 2023-11-28T11:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:12:43.104 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-11-28T11:12:41.624414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":80688,"kb_used_data":7472,"kb_used_omap":0,"kb_used_meta":73152,"kb_avail":754894032,"statfs":{"total":773094113280,"available":773011488768,"internally_reserved":0,"allocated":7651328,"data_stored":4219578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74907648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001359"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.241155+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:12:26.241155+0000","last_peered":"2023-11-28T11:12:26.241155+0000","last_clean":"2023-11-28T11:12:26.241155+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:12:26.241155+0000","last_undegraded":"2023-11-28T11:12:26.241155+0000","last_fullsized":"2023-11-28T11:12:26.241155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:27.108867+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:12:27.108867+0000","last_peered":"2023-11-28T11:12:27.108867+0000","last_clean":"2023-11-28T11:12:27.108867+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:12:27.108867+0000","last_undegraded":"2023-11-28T11:12:27.108867+0000","last_fullsized":"2023-11-28T11:12:27.108867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:25.960404+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:12:25.960404+0000","last_peered":"2023-11-28T11:12:25.960404+0000","last_clean":"2023-11-28T11:12:25.960404+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:12:25.960404+0000","last_undegraded":"2023-11-28T11:12:25.960404+0000","last_fullsized":"2023-11-28T11:12:25.960404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":201,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:33.846534+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:12:33.846534+0000","last_peered":"2023-11-28T11:12:33.846534+0000","last_clean":"2023-11-28T11:12:33.846534+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:12:33.846534+0000","last_undegraded":"2023-11-28T11:12:33.846534+0000","last_fullsized":"2023-11-28T11:12:33.846534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:27.109239+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:12:27.109239+0000","last_peered":"2023-11-28T11:12:27.109239+0000","last_clean":"2023-11-28T11:12:27.109239+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:12:27.109239+0000","last_undegraded":"2023-11-28T11:12:27.109239+0000","last_fullsized":"2023-11-28T11:12:27.109239+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:33.846495+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:12:33.846495+0000","last_peered":"2023-11-28T11:12:33.846495+0000","last_clean":"2023-11-28T11:12:33.846495+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:12:33.846495+0000","last_undegraded":"2023-11-28T11:12:33.846495+0000","last_fullsized":"2023-11-28T11:12:33.846495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:25.962068+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:12:25.962068+0000","last_peered":"2023-11-28T11:12:25.962068+0000","last_clean":"2023-11-28T11:12:25.962068+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:12:25.962068+0000","last_undegraded":"2023-11-28T11:12:25.962068+0000","last_fullsized":"2023-11-28T11:12:25.962068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.379752+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:12:26.379752+0000","last_peered":"2023-11-28T11:12:26.379752+0000","last_clean":"2023-11-28T11:12:26.379752+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:12:26.379752+0000","last_undegraded":"2023-11-28T11:12:26.379752+0000","last_fullsized":"2023-11-28T11:12:26.379752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:26.379975+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:12:26.379975+0000","last_peered":"2023-11-28T11:12:26.379975+0000","last_clean":"2023-11-28T11:12:26.379975+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:12:26.379975+0000","last_undegraded":"2023-11-28T11:12:26.379975+0000","last_fullsized":"2023-11-28T11:12:26.379975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10912,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360928,"statfs":{"total":96636764160,"available":96625590272,"internally_reserved":0,"allocated":1400832,"data_stored":968021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.746,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.46700000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54200000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.48599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.442},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":375784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70199999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":1.544,"15min":1.544},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66000000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.499},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":1.665,"15min":1.665},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.70599999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69599999999999995,"5min":1.762,"15min":1.762},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.39800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.002,"15min":1.002},"last":0.438},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10868,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360972,"statfs":{"total":96636764160,"available":96625635328,"internally_reserved":0,"allocated":1421312,"data_stored":979809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.247,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54900000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"min":{"1min":0.25800000000000001,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.251,"5min":0.187,"15min":0.187},"max":{"1min":0.54400000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39800000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63800000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.627,"5min":1.363,"15min":1.363},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.622}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90600000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":1.48,"15min":1.48},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.498},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.42299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":794624,"data_stored":375692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.222},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.33300000000000002},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.248}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55600000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53000000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.312},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57499999999999996,"5min":0.871,"15min":0.871},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.379}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80100000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68600000000000005,"15min":0.754},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.877,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.434},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.45400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9768,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362072,"statfs":{"total":96636764160,"available":96626761728,"internally_reserved":0,"allocated":819200,"data_stored":388841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33300000000000002},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.313}]},{"osd":1,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32700000000000001},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.247},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.628,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.51000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71699999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79100000000000004,"5min":1.024,"15min":1.024},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80500000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.76600000000000001,"5min":1.05,"15min":1.05},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9884,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361956,"statfs":{"total":96636764160,"available":96626642944,"internally_reserved":0,"allocated":806912,"data_stored":377165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46800000000000003,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.222},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.48999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59399999999999997,"5min":1.054,"15min":1.054},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56299999999999994,"5min":1.147,"15min":1.147},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55700000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.49299999999999999,"15min":0.59099999999999997},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":5.133,"15min":5.133},"last":0.34000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.499},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.67500000000000004,"5min":0.877,"15min":0.877},"last":0.628}]},{"osd":5,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":1.194,"15min":1.194},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":1.077,"15min":1.077},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":802816,"data_stored":377123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.505}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.65800000000000003,"5min":0.753,"15min":0.753},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.54500000000000004,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72199999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":377143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.73799999999999999,"15min":0.83599999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.76700000000000002,"15min":0.873},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69399999999999995,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64400000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.70899999999999996,"15min":0.78900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":6.601,"15min":6.601},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71399999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.45100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.83499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:43.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.226+0000 7f1408327700 1 Processor -- start 2023-11-28T11:12:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.226+0000 7f1408327700 1 -- start start 2023-11-28T11:12:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.226+0000 7f1408327700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f140015cd10 0x7f14000ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.226+0000 7f1408327700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f140015d0e0 0x7f14000cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.226+0000 7f1408327700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f14000b18a0 0x7f14000b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f1408327700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14000b2280 con 0x7f140015cd10 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f1408327700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14000cbfb0 con 0x7f14000b18a0 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f1408327700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000cb4e0 con 0x7f140015d0e0 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f140015cd10 0x7f14000ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f140015cd10 0x7f14000ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53652/0 (socket says 172.21.15.83:53652) 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 -- 172.21.15.83:0/1039834805 learned_addr learned my addr 172.21.15.83:0/1039834805 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14058c2700 1 --2- 172.21.15.83:0/1039834805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f140015d0e0 0x7f14000cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14068c4700 1 --2- 172.21.15.83:0/1039834805 >> v2:172.21.15.90:3300/0 conn(0x7f14000b18a0 0x7f14000b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 -- 172.21.15.83:0/1039834805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f140015d0e0 msgr2=0x7f14000cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 --2- 172.21.15.83:0/1039834805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f140015d0e0 0x7f14000cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 -- 172.21.15.83:0/1039834805 >> v2:172.21.15.90:3300/0 conn(0x7f14000b18a0 msgr2=0x7f14000b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 --2- 172.21.15.83:0/1039834805 >> v2:172.21.15.90:3300/0 conn(0x7f14000b18a0 0x7f14000b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 -- 172.21.15.83:0/1039834805 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000cb710 con 0x7f140015cd10 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14060c3700 1 --2- 172.21.15.83:0/1039834805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015cd10 0x7f14000ca970 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1400160910 tx=0x7f13f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4b5583c48d1a1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14050c1700 1 -- 172.21.15.83:0/1039834805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0011830 con 0x7f140015cd10 2023-11-28T11:12:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14050c1700 1 -- 172.21.15.83:0/1039834805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f0011990 con 0x7f140015cd10 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.227+0000 7f14050c1700 1 -- 172.21.15.83:0/1039834805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0011d20 con 0x7f140015cd10 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 -- 172.21.15.83:0/1039834805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015cd10 msgr2=0x7f14000ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 --2- 172.21.15.83:0/1039834805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015cd10 0x7f14000ca970 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1400160910 tx=0x7f13f0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 -- 172.21.15.83:0/1039834805 shutdown_connections 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 --2- 172.21.15.83:0/1039834805 >> v2:172.21.15.90:3300/0 conn(0x7f14000b18a0 0x7f14000b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 --2- 172.21.15.83:0/1039834805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015cd10 0x7f14000ca970 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 --2- 172.21.15.83:0/1039834805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f140015d0e0 0x7f14000cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 -- 172.21.15.83:0/1039834805 >> 172.21.15.83:0/1039834805 conn(0x7f140006e9a0 msgr2=0x7f14000cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 -- 172.21.15.83:0/1039834805 shutdown_connections 2023-11-28T11:12:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.228+0000 7f1408327700 1 -- 172.21.15.83:0/1039834805 wait complete. 2023-11-28T11:12:43.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.229+0000 7f1408327700 1 Processor -- start 2023-11-28T11:12:43.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.229+0000 7f1408327700 1 -- start start 2023-11-28T11:12:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f1408327700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14000b18a0 0x7f14001f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f1408327700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f140015cd10 0x7f14000c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f1408327700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 0x7f14000c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f1408327700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14001f24d0 con 0x7f140015d0e0 2023-11-28T11:12:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f1408327700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000c5450 con 0x7f140015cd10 2023-11-28T11:12:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f1408327700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1400155630 con 0x7f14000b18a0 2023-11-28T11:12:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 0x7f14000c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14058c2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f140015cd10 0x7f14000c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 0x7f14000c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53662/0 (socket says 172.21.15.83:53662) 2023-11-28T11:12:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 -- 172.21.15.83:0/2555859462 learned_addr learned my addr 172.21.15.83:0/2555859462 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14060c3700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14000b18a0 0x7f14001f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 -- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14000b18a0 msgr2=0x7f14001f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14000b18a0 0x7f14001f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 -- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f140015cd10 msgr2=0x7f14000c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f140015cd10 0x7f14000c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 -- 172.21.15.83:0/2555859462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009550 con 0x7f140015d0e0 2023-11-28T11:12:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.230+0000 7f14068c4700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 0x7f14000c6ef0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13f800eed0 tx=0x7f13f800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.231+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8010040 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.231+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f800bc80 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.231+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8009e40 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.231+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14000cbcd0 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.231+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14000cb9f0 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.232+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140006b9b0 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.232+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13f8014400 con 0x7f140015d0e0 2023-11-28T11:12:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.232+0000 7f13f6ffd700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13e40682f0 0x7f13e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.233+0000 7f14060c3700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13e40682f0 0x7f13e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.233+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f13f8085fa0 con 0x7f140015d0e0 2023-11-28T11:12:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.233+0000 7f14060c3700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13e40682f0 0x7f13e406a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009c10 tx=0x7f13f0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.237+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13f8052960 con 0x7f140015d0e0 2023-11-28T11:12:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.422+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14000b27b0 con 0x7f140015d0e0 2023-11-28T11:12:43.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.423+0000 7f13f6ffd700 1 -- 172.21.15.83:0/2555859462 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+14496 (secure 0 0 0) 0x7f13f8018020 con 0x7f140015d0e0 2023-11-28T11:12:43.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:43.424 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":68,"fsid":"69424f07-94c3-4dbc-8507-3551775e0a09","created":"2023-11-28T11:08:18.043375+0000","modified":"2023-11-28T11:12:40.048249+0000","last_up_change":"2023-11-28T11:08:28.805065+0000","last_in_change":"2023-11-28T11:08:20.553982+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:29.338657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:29.610310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b3e54911-ebed-4e23-9277-fae35bddfa94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":103534},{"type":"v1","addr":"172.21.15.83:6801","nonce":103534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":103534},{"type":"v1","addr":"172.21.15.83:6803","nonce":103534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":103534},{"type":"v1","addr":"172.21.15.83:6807","nonce":103534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":103534},{"type":"v1","addr":"172.21.15.83:6805","nonce":103534}]},"public_addr":"172.21.15.83:6801/103534","cluster_addr":"172.21.15.83:6803/103534","heartbeat_back_addr":"172.21.15.83:6807/103534","heartbeat_front_addr":"172.21.15.83:6805/103534","state":["exists","up"]},{"osd":1,"uuid":"b3ea33d8-2165-4137-bf95-4e525e5171e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":103535},{"type":"v1","addr":"172.21.15.83:6811","nonce":103535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":103535},{"type":"v1","addr":"172.21.15.83:6816","nonce":103535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":103535},{"type":"v1","addr":"172.21.15.83:6823","nonce":103535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":103535},{"type":"v1","addr":"172.21.15.83:6821","nonce":103535}]},"public_addr":"172.21.15.83:6811/103535","cluster_addr":"172.21.15.83:6816/103535","heartbeat_back_addr":"172.21.15.83:6823/103535","heartbeat_front_addr":"172.21.15.83:6821/103535","state":["exists","up"]},{"osd":2,"uuid":"30f1202f-b122-4cf7-a737-7d71678bee11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":103544},{"type":"v1","addr":"172.21.15.83:6825","nonce":103544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":103544},{"type":"v1","addr":"172.21.15.83:6827","nonce":103544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":103544},{"type":"v1","addr":"172.21.15.83:6831","nonce":103544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":103544},{"type":"v1","addr":"172.21.15.83:6829","nonce":103544}]},"public_addr":"172.21.15.83:6825/103544","cluster_addr":"172.21.15.83:6827/103544","heartbeat_back_addr":"172.21.15.83:6831/103544","heartbeat_front_addr":"172.21.15.83:6829/103544","state":["exists","up"]},{"osd":3,"uuid":"1f803565-3bcc-4627-8bd4-157d7a1be074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":103549},{"type":"v1","addr":"172.21.15.83:6810","nonce":103549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":103549},{"type":"v1","addr":"172.21.15.83:6813","nonce":103549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":103549},{"type":"v1","addr":"172.21.15.83:6820","nonce":103549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":103549},{"type":"v1","addr":"172.21.15.83:6817","nonce":103549}]},"public_addr":"172.21.15.83:6810/103549","cluster_addr":"172.21.15.83:6813/103549","heartbeat_back_addr":"172.21.15.83:6820/103549","heartbeat_front_addr":"172.21.15.83:6817/103549","state":["exists","up"]},{"osd":4,"uuid":"59c878b2-da83-47b2-a5fb-fdf4fefc14b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":103766},{"type":"v1","addr":"172.21.15.90:6817","nonce":103766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":103766},{"type":"v1","addr":"172.21.15.90:6819","nonce":103766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":103766},{"type":"v1","addr":"172.21.15.90:6823","nonce":103766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":103766},{"type":"v1","addr":"172.21.15.90:6821","nonce":103766}]},"public_addr":"172.21.15.90:6817/103766","cluster_addr":"172.21.15.90:6819/103766","heartbeat_back_addr":"172.21.15.90:6823/103766","heartbeat_front_addr":"172.21.15.90:6821/103766","state":["exists","up"]},{"osd":5,"uuid":"d04ea464-897d-4c86-b799-f04692ae0f16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":103767},{"type":"v1","addr":"172.21.15.90:6825","nonce":103767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":103767},{"type":"v1","addr":"172.21.15.90:6827","nonce":103767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":103767},{"type":"v1","addr":"172.21.15.90:6831","nonce":103767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":103767},{"type":"v1","addr":"172.21.15.90:6829","nonce":103767}]},"public_addr":"172.21.15.90:6825/103767","cluster_addr":"172.21.15.90:6827/103767","heartbeat_back_addr":"172.21.15.90:6831/103767","heartbeat_front_addr":"172.21.15.90:6829/103767","state":["exists","up"]},{"osd":6,"uuid":"9a9bb320-9fc5-4038-a54a-93ca2fe9200b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":103764},{"type":"v1","addr":"172.21.15.90:6801","nonce":103764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":103764},{"type":"v1","addr":"172.21.15.90:6803","nonce":103764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":103764},{"type":"v1","addr":"172.21.15.90:6807","nonce":103764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":103764},{"type":"v1","addr":"172.21.15.90:6805","nonce":103764}]},"public_addr":"172.21.15.90:6801/103764","cluster_addr":"172.21.15.90:6803/103764","heartbeat_back_addr":"172.21.15.90:6807/103764","heartbeat_front_addr":"172.21.15.90:6805/103764","state":["exists","up"]},{"osd":7,"uuid":"646d177e-e2dc-4ed0-95b1-724a20d6aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":103765},{"type":"v1","addr":"172.21.15.90:6809","nonce":103765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":103765},{"type":"v1","addr":"172.21.15.90:6811","nonce":103765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":103765},{"type":"v1","addr":"172.21.15.90:6815","nonce":103765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":103765},{"type":"v1","addr":"172.21.15.90:6813","nonce":103765}]},"public_addr":"172.21.15.90:6809/103765","cluster_addr":"172.21.15.90:6811/103765","heartbeat_back_addr":"172.21.15.90:6815/103765","heartbeat_front_addr":"172.21.15.90:6813/103765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.762508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.732554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.738346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.739619+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.722049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:28.055646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.554981+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:25.737855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3038492821":"2023-11-29T11:12:36.709159+0000","172.21.15.83:6835/3038492821":"2023-11-29T11:12:36.709159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13e40682f0 msgr2=0x7f13e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13e40682f0 0x7f13e406a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009c10 tx=0x7f13f0008030 comp rx=0 tx=0).stop 2023-11-28T11:12:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 msgr2=0x7f14000c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 0x7f14000c6ef0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13f800eed0 tx=0x7f13f800b390 comp rx=0 tx=0).stop 2023-11-28T11:12:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 shutdown_connections 2023-11-28T11:12:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f13e40682f0 0x7f13e406a7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140015d0e0 0x7f14000c6ef0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f14000b18a0 0x7f14001f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 --2- 172.21.15.83:0/2555859462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f140015cd10 0x7f14000c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 >> 172.21.15.83:0/2555859462 conn(0x7f140006e9a0 msgr2=0x7f140015be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 shutdown_connections 2023-11-28T11:12:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:43.425+0000 7f1408327700 1 -- 172.21.15.83:0/2555859462 wait complete. 2023-11-28T11:12:44.438 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T11:12:44.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.576+0000 7fb4198b8700 1 Processor -- start 2023-11-28T11:12:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.576+0000 7fb4198b8700 1 -- start start 2023-11-28T11:12:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4198b8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 0x7fb41415d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4198b8700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb41406ad10 0x7fb4140b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4198b8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41406a270 con 0x7fb4140b4f80 2023-11-28T11:12:44.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4198b8700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41415dc90 con 0x7fb41406ad10 2023-11-28T11:12:44.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4198b8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b5400 con 0x7fb41415a230 2023-11-28T11:12:44.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4137fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb4140b4f80 0x7fb4140d0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51530/0 (socket says 172.21.15.83:51530) 2023-11-28T11:12:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4137fe700 1 -- 172.21.15.83:0/1166267396 learned_addr learned my addr 172.21.15.83:0/1166267396 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb4127fc700 1 --2- 172.21.15.83:0/1166267396 >> v2:172.21.15.90:3300/0 conn(0x7fb41406ad10 0x7fb4140b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb412ffd700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 0x7fb41415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772915204 0 0) 0x7fb41406a270 con 0x7fb4140b4f80 2023-11-28T11:12:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc0035b0 con 0x7fb4140b4f80 2023-11-28T11:12:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.577+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470454171 0 0) 0x7fb3fc0035b0 con 0x7fb4140b4f80 2023-11-28T11:12:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 msgr2=0x7fb41415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 0x7fb41415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 >> v2:172.21.15.90:3300/0 conn(0x7fb41406ad10 msgr2=0x7fb4140b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:44.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 --2- 172.21.15.83:0/1166267396 >> v2:172.21.15.90:3300/0 conn(0x7fb41406ad10 0x7fb4140b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140cd9d0 con 0x7fb4140b4f80 2023-11-28T11:12:44.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 --2- 172.21.15.83:0/1166267396 >> v2:172.21.15.90:3300/0 conn(0x7fb41406ad10 0x7fb4140b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fb40c0019f0 con 0x7fb4140b4f80 2023-11-28T11:12:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 >> v1:172.21.15.83:6789/0 conn(0x7fb4140b4f80 legacy=0x7fb4140d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 0x7fb3fc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3fc003f80 0x7fb3fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3fc004d90 0x7fb3fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41415dc90 con 0x7fb3fc003f80 2023-11-28T11:12:44.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b5400 con 0x7fb3fc004d90 2023-11-28T11:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc00ba10 con 0x7fb41415a230 2023-11-28T11:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3fc003f80 0x7fb3fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb412ffd700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 0x7fb3fc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4137fe700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3fc004d90 0x7fb3fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 -- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 msgr2=0x7fb3fc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb41415a230 0x7fb3fc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 -- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3fc004d90 msgr2=0x7fb3fc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3fc004d90 0x7fb3fc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 -- 172.21.15.83:0/1166267396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0035b0 con 0x7fb3fc003f80 2023-11-28T11:12:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4137fe700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3fc004d90 0x7fb3fc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb4127fc700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3fc003f80 0x7fb3fc004850 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb404009d30 tx=0x7fb3fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7938bd4d15283a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404009e80 con 0x7fb3fc003f80 2023-11-28T11:12:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40400e9a0 con 0x7fb3fc003f80 2023-11-28T11:12:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.578+0000 7fb411ffb700 1 -- 172.21.15.83:0/1166267396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40400ed30 con 0x7fb3fc003f80 2023-11-28T11:12:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4198b8700 1 -- 172.21.15.83:0/1166267396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3fc003f80 msgr2=0x7fb3fc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4198b8700 1 --2- 172.21.15.83:0/1166267396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3fc003f80 0x7fb3fc004850 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb404009d30 tx=0x7fb3fc005880 comp rx=0 tx=0).stop 2023-11-28T11:12:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4137fe700 1 -- 172.21.15.83:0/1166267396 reap_dead start 2023-11-28T11:12:44.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4198b8700 1 -- 172.21.15.83:0/1166267396 shutdown_connections 2023-11-28T11:12:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4198b8700 1 -- 172.21.15.83:0/1166267396 >> 172.21.15.83:0/1166267396 conn(0x7fb4140d2fc0 msgr2=0x7fb4140d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4198b8700 1 -- 172.21.15.83:0/1166267396 shutdown_connections 2023-11-28T11:12:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.579+0000 7fb4198b8700 1 -- 172.21.15.83:0/1166267396 wait complete. 2023-11-28T11:12:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.580+0000 7fb4198b8700 1 Processor -- start 2023-11-28T11:12:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.581+0000 7fb4198b8700 1 -- start start 2023-11-28T11:12:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb3fc003f80 0x7fb4140c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb41415e8c0 0x7fb4140c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4140c8310 0x7fb4140c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41415f940 con 0x7fb4140c8310 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140c8cc0 con 0x7fb41415e8c0 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41415edc0 con 0x7fb3fc003f80 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4140c8310 0x7fb4140c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4140c8310 0x7fb4140c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53682/0 (socket says 172.21.15.83:53682) 2023-11-28T11:12:44.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 -- 172.21.15.83:0/693920691 learned_addr learned my addr 172.21.15.83:0/693920691 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4127fc700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb41415e8c0 0x7fb4140c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb412ffd700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb3fc003f80 0x7fb4140c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 -- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb3fc003f80 msgr2=0x7fb4140c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb3fc003f80 0x7fb4140c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 -- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb41415e8c0 msgr2=0x7fb4140c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb41415e8c0 0x7fb4140c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 -- 172.21.15.83:0/693920691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41406a510 con 0x7fb4140c8310 2023-11-28T11:12:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4127fc700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb41415e8c0 0x7fb4140c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4137fe700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4140c8310 0x7fb4140c8780 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb4140b90f0 tx=0x7fb40c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00ee90 con 0x7fb4140c8310 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c0019e0 con 0x7fb4140c8310 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4140cdcb0 con 0x7fb4140c8310 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.582+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c013660 con 0x7fb4140c8310 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.583+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41406a7f0 con 0x7fb4140c8310 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.583+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb414152d20 con 0x7fb4140c8310 2023-11-28T11:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb40c0137c0 con 0x7fb4140c8310 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb3fbfff700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb4000682f0 0x7fb40006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb40c086cb0 con 0x7fb4140c8310 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb412ffd700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb4000682f0 0x7fb40006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb3fbfff700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7fb40006eb90 0x7fb400070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 --> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4000716b0 con 0x7fb40006eb90 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.584+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb40c0870f0 con 0x7fb4140c8310 2023-11-28T11:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.585+0000 7fb4127fc700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7fb40006eb90 0x7fb400070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.585+0000 7fb412ffd700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb4000682f0 0x7fb40006a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb408009b80 tx=0x7fb408009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.585+0000 7fb4127fc700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7fb40006eb90 0x7fb400070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.585+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== osd.0 v2:172.21.15.83:6800/103534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb4000716b0 con 0x7fb40006eb90 2023-11-28T11:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 --> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb4140742b0 con 0x7fb40006eb90 2023-11-28T11:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb3fbfff700 1 -- 172.21.15.83:0/693920691 <== osd.0 v2:172.21.15.83:6800/103534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb4140742b0 con 0x7fb40006eb90 2023-11-28T11:12:44.612 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:44.612 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:44.613 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7fb40006eb90 msgr2=0x7fb400070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6800/103534,v1:172.21.15.83:6801/103534] conn(0x7fb40006eb90 0x7fb400070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb4000682f0 msgr2=0x7fb40006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb4000682f0 0x7fb40006a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb408009b80 tx=0x7fb408009210 comp rx=0 tx=0).stop 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4140c8310 msgr2=0x7fb4140c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 --2- 172.21.15.83:0/693920691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4140c8310 0x7fb4140c8780 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb4140b90f0 tx=0x7fb40c00e720 comp rx=0 tx=0).stop 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4137fe700 1 -- 172.21.15.83:0/693920691 reap_dead start 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 shutdown_connections 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 >> 172.21.15.83:0/693920691 conn(0x7fb4140d2fc0 msgr2=0x7fb4140b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 shutdown_connections 2023-11-28T11:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.600+0000 7fb4198b8700 1 -- 172.21.15.83:0/693920691 wait complete. 2023-11-28T11:12:44.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T11:12:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.736+0000 7f40ce2eb700 1 Processor -- start 2023-11-28T11:12:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.736+0000 7f40ce2eb700 1 -- start start 2023-11-28T11:12:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40ce2eb700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c80b1900 0x7f40c80b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40ce2eb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f40c815e000 0x7f40c815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40ce2eb700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c806bc10 con 0x7f40c80b5600 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40ce2eb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c806cc20 con 0x7f40c815e000 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40ce2eb700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c806be60 con 0x7f40c80b1900 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f40c815e000 0x7f40c815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f40c815e000 0x7f40c815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53692/0 (socket says 172.21.15.83:53692) 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 -- 172.21.15.83:0/1919534552 learned_addr learned my addr 172.21.15.83:0/1919534552 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40c7fff700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c80b1900 0x7f40c80b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 -- 172.21.15.83:0/1919534552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c80b1900 msgr2=0x7f40c80b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c80b1900 0x7f40c80b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 -- 172.21.15.83:0/1919534552 >> v1:172.21.15.90:6789/0 conn(0x7f40c80b5600 legacy=0x7f40c80cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 -- 172.21.15.83:0/1919534552 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c806c140 con 0x7f40c815e000 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40c7fff700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c80b1900 0x7f40c80b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.737+0000 7f40cc888700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c815e000 0x7f40c815a5c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009e50 tx=0x7f40c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd56f17ab8b7948e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40c6ffd700 1 -- 172.21.15.83:0/1919534552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000e070 con 0x7f40c815e000 2023-11-28T11:12:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40c6ffd700 1 -- 172.21.15.83:0/1919534552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c0003a20 con 0x7f40c815e000 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40c6ffd700 1 -- 172.21.15.83:0/1919534552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0003dd0 con 0x7f40c815e000 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 -- 172.21.15.83:0/1919534552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c815e000 msgr2=0x7f40c815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c815e000 0x7f40c815a5c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009e50 tx=0x7f40c000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 -- 172.21.15.83:0/1919534552 shutdown_connections 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c815e000 0x7f40c815a5c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/1919534552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c80b1900 0x7f40c80b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 -- 172.21.15.83:0/1919534552 >> 172.21.15.83:0/1919534552 conn(0x7f40c80cc690 msgr2=0x7f40c80cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 -- 172.21.15.83:0/1919534552 shutdown_connections 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.738+0000 7f40ce2eb700 1 -- 172.21.15.83:0/1919534552 wait complete. 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.739+0000 7f40ce2eb700 1 Processor -- start 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 -- start start 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 0x7f40c81eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c815e000 0x7f40c80c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c80c3720 0x7f40c80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c3c00 con 0x7f40c80b1900 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c3ee0 con 0x7f40c80c3720 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40ce2eb700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c41c0 con 0x7f40c815e000 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.740+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 0x7f40c81eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 0x7f40c81eebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53694/0 (socket says 172.21.15.83:53694) 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 -- 172.21.15.83:0/2779657599 learned_addr learned my addr 172.21.15.83:0/2779657599 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40cc888700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c80c3720 0x7f40c80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c77fe700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c815e000 0x7f40c80c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 -- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c815e000 msgr2=0x7f40c80c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c815e000 0x7f40c80c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 -- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c80c3720 msgr2=0x7f40c80b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c80c3720 0x7f40c80b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 -- 172.21.15.83:0/2779657599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009b30 con 0x7f40c80b1900 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c7fff700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 0x7f40c81eebb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00c750 tx=0x7f40bc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc011800 con 0x7f40c80b1900 2023-11-28T11:12:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc011960 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c806c700 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc011cd0 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.741+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c806c420 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.742+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c815ab00 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.742+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f40bc020070 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.743+0000 7f40c4ff9700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40b40682f0 0x7f40b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.743+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f40bc085950 con 0x7f40c80b1900 2023-11-28T11:12:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.743+0000 7f40c77fe700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40b40682f0 0x7f40b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.743+0000 7f40c77fe700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40b40682f0 0x7f40b406a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f40c80b1660 tx=0x7f40b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.747+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40bc052370 con 0x7f40c80b1900 2023-11-28T11:12:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.929+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f40c80c4330 con 0x7f40b40682f0 2023-11-28T11:12:44.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.930+0000 7f40c4ff9700 1 -- 172.21.15.83:0/2779657599 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f40c80c4330 con 0x7f40b40682f0 2023-11-28T11:12:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40b40682f0 msgr2=0x7f40b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40b40682f0 0x7f40b406a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f40c80b1660 tx=0x7f40b8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 msgr2=0x7f40c81eebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 0x7f40c81eebb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00c750 tx=0x7f40bc00cb10 comp rx=0 tx=0).stop 2023-11-28T11:12:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 shutdown_connections 2023-11-28T11:12:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c80c3720 0x7f40c80b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f40b40682f0 0x7f40b406a7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c80b1900 0x7f40c81eebb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 --2- 172.21.15.83:0/2779657599 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40c815e000 0x7f40c80c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:44.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 >> 172.21.15.83:0/2779657599 conn(0x7f40c80cc690 msgr2=0x7f40c80ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:44.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 shutdown_connections 2023-11-28T11:12:44.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:44.932+0000 7f40ce2eb700 1 -- 172.21.15.83:0/2779657599 wait complete. 2023-11-28T11:12:44.936 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T11:12:44.944 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T11:12:44.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 Processor -- start 2023-11-28T11:12:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 -- start start 2023-11-28T11:12:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29580b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f29580b5670 0x7f29580b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29580b1c60 0x7f29580cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29580b5ae0 con 0x7f29580b1c60 2023-11-28T11:12:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29580cb3a0 con 0x7f29580b5670 2023-11-28T11:12:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.069+0000 7f295d319700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580cb640 con 0x7f29581599b0 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29580b1c60 0x7f29580cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29580b1c60 0x7f29580cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53704/0 (socket says 172.21.15.83:53704) 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 -- 172.21.15.83:0/818321178 learned_addr learned my addr 172.21.15.83:0/818321178 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29567fc700 1 --2- 172.21.15.83:0/818321178 >> v2:172.21.15.90:3300/0 conn(0x7f29580b5670 0x7f29580b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f2956ffd700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29580b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 -- 172.21.15.83:0/818321178 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 msgr2=0x7f29580b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29580b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 -- 172.21.15.83:0/818321178 >> v2:172.21.15.90:3300/0 conn(0x7f29580b5670 msgr2=0x7f29580b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 --2- 172.21.15.83:0/818321178 >> v2:172.21.15.90:3300/0 conn(0x7f29580b5670 0x7f29580b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 -- 172.21.15.83:0/818321178 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580cb920 con 0x7f29580b1c60 2023-11-28T11:12:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f2956ffd700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29580b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.070+0000 7f29577fe700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b1c60 0x7f29580cd7e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f295806db70 tx=0x7f295000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696524f0b925735f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f2955ffb700 1 -- 172.21.15.83:0/818321178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950010070 con 0x7f29580b1c60 2023-11-28T11:12:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f2955ffb700 1 -- 172.21.15.83:0/818321178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295000dce0 con 0x7f29580b1c60 2023-11-28T11:12:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f2955ffb700 1 -- 172.21.15.83:0/818321178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000b310 con 0x7f29580b1c60 2023-11-28T11:12:45.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 -- 172.21.15.83:0/818321178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b1c60 msgr2=0x7f29580cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b1c60 0x7f29580cd7e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f295806db70 tx=0x7f295000d430 comp rx=0 tx=0).stop 2023-11-28T11:12:45.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 -- 172.21.15.83:0/818321178 shutdown_connections 2023-11-28T11:12:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29580b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 --2- 172.21.15.83:0/818321178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b1c60 0x7f29580cd7e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 --2- 172.21.15.83:0/818321178 >> v2:172.21.15.90:3300/0 conn(0x7f29580b5670 0x7f29580b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 -- 172.21.15.83:0/818321178 >> 172.21.15.83:0/818321178 conn(0x7f29580cc690 msgr2=0x7f29580cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 -- 172.21.15.83:0/818321178 shutdown_connections 2023-11-28T11:12:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.074+0000 7f295d319700 1 -- 172.21.15.83:0/818321178 wait complete. 2023-11-28T11:12:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.076+0000 7f295d319700 1 Processor -- start 2023-11-28T11:12:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.076+0000 7f295d319700 1 -- start start 2023-11-28T11:12:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f295d319700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29580b1c60 0x7f2958151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f295d319700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b5670 0x7f29581e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f295d319700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29581e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f295d319700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29581e4360 con 0x7f29580b5670 2023-11-28T11:12:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f295d319700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958152430 con 0x7f29580b1c60 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f295d319700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580c38c0 con 0x7f29581599b0 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f29567fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b5670 0x7f29581e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f29567fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b5670 0x7f29581e37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53710/0 (socket says 172.21.15.83:53710) 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f29567fc700 1 -- 172.21.15.83:0/1681983552 learned_addr learned my addr 172.21.15.83:0/1681983552 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f29577fe700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29581e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.077+0000 7f2956ffd700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29580b1c60 0x7f2958151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29567fc700 1 -- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 msgr2=0x7f29581e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29567fc700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29581e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29567fc700 1 -- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29580b1c60 msgr2=0x7f2958151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29567fc700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29580b1c60 0x7f2958151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29567fc700 1 -- 172.21.15.83:0/1681983552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295000bd40 con 0x7f29580b5670 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29577fe700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29581599b0 0x7f29581e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f29567fc700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b5670 0x7f29581e37f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e930 tx=0x7f294c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009690 con 0x7f29580b5670 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c00c850 con 0x7f29580b5670 2023-11-28T11:12:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29580cbee0 con 0x7f29580b5670 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c00cbe0 con 0x7f29580b5670 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.078+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580cbc00 con 0x7f29580b5670 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.079+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29580cc1c0 con 0x7f29580b5670 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.079+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f294c019030 con 0x7f29580b5670 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.079+0000 7f293f7fe700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29440682f0 0x7f294406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f294c085b60 con 0x7f29580b5670 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f2956ffd700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29440682f0 0x7f294406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f293f7fe700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f294406eb90 0x7f2944070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 --> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29440716b0 con 0x7f294406eb90 2023-11-28T11:12:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294c08a2a0 con 0x7f29580b5670 2023-11-28T11:12:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f29577fe700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f294406eb90 0x7f2944070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f2956ffd700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29440682f0 0x7f294406a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2948009930 tx=0x7f2948008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.080+0000 7f29577fe700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f294406eb90 0x7f2944070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.081+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== osd.1 v2:172.21.15.83:6809/103535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f29440716b0 con 0x7f294406eb90 2023-11-28T11:12:45.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.095+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 --> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f29580c3a30 con 0x7f294406eb90 2023-11-28T11:12:45.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.095+0000 7f293f7fe700 1 -- 172.21.15.83:0/1681983552 <== osd.1 v2:172.21.15.83:6809/103535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f29580c3a30 con 0x7f294406eb90 2023-11-28T11:12:45.096 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:45.096 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:45.096 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:45.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.095+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f294406eb90 msgr2=0x7f2944070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.095+0000 7f295d319700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6809/103535,v1:172.21.15.83:6811/103535] conn(0x7f294406eb90 0x7f2944070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.095+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29440682f0 msgr2=0x7f294406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.095+0000 7f295d319700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f29440682f0 0x7f294406a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2948009930 tx=0x7f2948008040 comp rx=0 tx=0).stop 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b5670 msgr2=0x7f29581e37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f295d319700 1 --2- 172.21.15.83:0/1681983552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29580b5670 0x7f29581e37f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e930 tx=0x7f294c00ecf0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f29577fe700 1 -- 172.21.15.83:0/1681983552 reap_dead start 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 shutdown_connections 2023-11-28T11:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 >> 172.21.15.83:0/1681983552 conn(0x7f29580cc690 msgr2=0x7f29580cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 shutdown_connections 2023-11-28T11:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.096+0000 7f295d319700 1 -- 172.21.15.83:0/1681983552 wait complete. 2023-11-28T11:12:45.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T11:12:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 Processor -- start 2023-11-28T11:12:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 -- start start 2023-11-28T11:12:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0eb415c890 0x7f0eb40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0eb40b4bb0 0x7f0eb406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb406e1a0 0x7f0eb41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb406e620 con 0x7f0eb40b4bb0 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb40b5020 con 0x7f0eb415c890 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.229+0000 7f0eb99ce700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb406d4a0 con 0x7f0eb406e1a0 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0eb40b4bb0 0x7f0eb406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0eb40b4bb0 0x7f0eb406dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53718/0 (socket says 172.21.15.83:53718) 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 -- 172.21.15.83:0/971559660 learned_addr learned my addr 172.21.15.83:0/971559660 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb37fe700 1 --2- 172.21.15.83:0/971559660 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb406e1a0 0x7f0eb41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 -- 172.21.15.83:0/971559660 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb406e1a0 msgr2=0x7f0eb41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 --2- 172.21.15.83:0/971559660 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb406e1a0 0x7f0eb41604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 -- 172.21.15.83:0/971559660 >> v2:172.21.15.90:3300/0 conn(0x7f0eb415c890 msgr2=0x7f0eb40aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb2ffd700 1 --2- 172.21.15.83:0/971559660 >> v2:172.21.15.90:3300/0 conn(0x7f0eb415c890 0x7f0eb40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 --2- 172.21.15.83:0/971559660 >> v2:172.21.15.90:3300/0 conn(0x7f0eb415c890 0x7f0eb40aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 -- 172.21.15.83:0/971559660 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40b5ae0 con 0x7f0eb40b4bb0 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb27fc700 1 --2- 172.21.15.83:0/971559660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb406dc60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40715b0 tx=0x7f0ea8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac2f833c1602133 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb1ffb700 1 -- 172.21.15.83:0/971559660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800b870 con 0x7f0eb40b4bb0 2023-11-28T11:12:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb1ffb700 1 -- 172.21.15.83:0/971559660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea800b9d0 con 0x7f0eb40b4bb0 2023-11-28T11:12:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.230+0000 7f0eb1ffb700 1 -- 172.21.15.83:0/971559660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800bd60 con 0x7f0eb40b4bb0 2023-11-28T11:12:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 -- 172.21.15.83:0/971559660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb40b4bb0 msgr2=0x7f0eb406dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/971559660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb406dc60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40715b0 tx=0x7f0ea8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 -- 172.21.15.83:0/971559660 shutdown_connections 2023-11-28T11:12:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/971559660 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb406e1a0 0x7f0eb41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/971559660 >> v2:172.21.15.90:3300/0 conn(0x7f0eb415c890 0x7f0eb40aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/971559660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb406dc60 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 -- 172.21.15.83:0/971559660 >> 172.21.15.83:0/971559660 conn(0x7f0eb40b2b60 msgr2=0x7f0eb40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 -- 172.21.15.83:0/971559660 shutdown_connections 2023-11-28T11:12:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.231+0000 7f0eb99ce700 1 -- 172.21.15.83:0/971559660 wait complete. 2023-11-28T11:12:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.232+0000 7f0eb99ce700 1 Processor -- start 2023-11-28T11:12:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.232+0000 7f0eb99ce700 1 -- start start 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 0x7f0eb41562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb415c890 0x7f0eb415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40c9f20 con 0x7f0eb406e1a0 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40ca170 con 0x7f0eb40b4bb0 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40ca450 con 0x7f0eb415c890 2023-11-28T11:12:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 0x7f0eb41562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 0x7f0eb41562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53724/0 (socket says 172.21.15.83:53724) 2023-11-28T11:12:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/2465247507 learned_addr learned my addr 172.21.15.83:0/2465247507 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb37fe700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb415c890 0x7f0eb415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb27fc700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb415c890 msgr2=0x7f0eb415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb415c890 0x7f0eb415ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0eb40b4bb0 msgr2=0x7f0eb415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 -- 172.21.15.83:0/2465247507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8009550 con 0x7f0eb406e1a0 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb37fe700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb415c890 0x7f0eb415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb2ffd700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 0x7f0eb41562a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c7f0 tx=0x7f0ea400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4011800 con 0x7f0eb406e1a0 2023-11-28T11:12:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea4011960 con 0x7f0eb406e1a0 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb40b5240 con 0x7f0eb406e1a0 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.233+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4011cf0 con 0x7f0eb406e1a0 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.234+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb406c400 con 0x7f0eb406e1a0 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.234+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb40b5520 con 0x7f0eb406e1a0 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.235+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ea4021070 con 0x7f0eb406e1a0 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.235+0000 7f0e9bfff700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea00682f0 0x7f0ea006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.235+0000 7f0eb27fc700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea00682f0 0x7f0ea006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.235+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0ea4085990 con 0x7f0eb406e1a0 2023-11-28T11:12:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.236+0000 7f0eb27fc700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea00682f0 0x7f0ea006a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009840 tx=0x7f0ea8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.240+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0ea4052200 con 0x7f0eb406e1a0 2023-11-28T11:12:45.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.422+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb40b3820 con 0x7f0ea00682f0 2023-11-28T11:12:45.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.422+0000 7f0e9bfff700 1 -- 172.21.15.83:0/2465247507 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0eb40b3820 con 0x7f0ea00682f0 2023-11-28T11:12:45.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea00682f0 msgr2=0x7f0ea006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea00682f0 0x7f0ea006a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009840 tx=0x7f0ea8008030 comp rx=0 tx=0).stop 2023-11-28T11:12:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 msgr2=0x7f0eb41562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 0x7f0eb41562a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c7f0 tx=0x7f0ea400cb00 comp rx=0 tx=0).stop 2023-11-28T11:12:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 shutdown_connections 2023-11-28T11:12:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0eb415c890 0x7f0eb415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f0ea00682f0 0x7f0ea006a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eb406e1a0 0x7f0eb41562a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 --2- 172.21.15.83:0/2465247507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0eb40b4bb0 0x7f0eb415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.424+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 >> 172.21.15.83:0/2465247507 conn(0x7f0eb40b2b60 msgr2=0x7f0eb40b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.425+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 shutdown_connections 2023-11-28T11:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.425+0000 7f0eb99ce700 1 -- 172.21.15.83:0/2465247507 wait complete. 2023-11-28T11:12:45.430 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T11:12:45.437 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T11:12:45.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:45.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.561+0000 7f815249a700 1 Processor -- start 2023-11-28T11:12:45.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.561+0000 7f815249a700 1 -- start start 2023-11-28T11:12:45.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f815249a700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f814c0ce140 0x7f814c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f815249a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f814c15c710 0x7f814c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f815249a700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c0ca2b0 con 0x7f814c071340 2023-11-28T11:12:45.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f815249a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c15ceb0 con 0x7f814c15c710 2023-11-28T11:12:45.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f815249a700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c15d190 con 0x7f814c0ce140 2023-11-28T11:12:45.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f814c15c710 0x7f814c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f814c15c710 0x7f814c0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53732/0 (socket says 172.21.15.83:53732) 2023-11-28T11:12:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 -- 172.21.15.83:0/1884649489 learned_addr learned my addr 172.21.15.83:0/1884649489 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814bfff700 1 --2- 172.21.15.83:0/1884649489 >> v2:172.21.15.90:3300/0 conn(0x7f814c0ce140 0x7f814c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 -- 172.21.15.83:0/1884649489 >> v2:172.21.15.90:3300/0 conn(0x7f814c0ce140 msgr2=0x7f814c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 --2- 172.21.15.83:0/1884649489 >> v2:172.21.15.90:3300/0 conn(0x7f814c0ce140 0x7f814c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 -- 172.21.15.83:0/1884649489 >> v1:172.21.15.90:6789/0 conn(0x7f814c071340 legacy=0x7f814c0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 -- 172.21.15.83:0/1884649489 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c15d470 con 0x7f814c15c710 2023-11-28T11:12:45.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814b7fe700 1 --2- 172.21.15.83:0/1884649489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c15c710 0x7f814c0cced0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f814c072000 tx=0x7f8140009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21214f49a00219 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814affd700 1 -- 172.21.15.83:0/1884649489 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140004230 con 0x7f814c15c710 2023-11-28T11:12:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814affd700 1 -- 172.21.15.83:0/1884649489 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8140004390 con 0x7f814c15c710 2023-11-28T11:12:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.562+0000 7f814affd700 1 -- 172.21.15.83:0/1884649489 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81400036a0 con 0x7f814c15c710 2023-11-28T11:12:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 -- 172.21.15.83:0/1884649489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c15c710 msgr2=0x7f814c0cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 --2- 172.21.15.83:0/1884649489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c15c710 0x7f814c0cced0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f814c072000 tx=0x7f8140009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 -- 172.21.15.83:0/1884649489 shutdown_connections 2023-11-28T11:12:45.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 --2- 172.21.15.83:0/1884649489 >> v2:172.21.15.90:3300/0 conn(0x7f814c0ce140 0x7f814c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 --2- 172.21.15.83:0/1884649489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c15c710 0x7f814c0cced0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 -- 172.21.15.83:0/1884649489 >> 172.21.15.83:0/1884649489 conn(0x7f814c0cf300 msgr2=0x7f814c0cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 -- 172.21.15.83:0/1884649489 shutdown_connections 2023-11-28T11:12:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.563+0000 7f815249a700 1 -- 172.21.15.83:0/1884649489 wait complete. 2023-11-28T11:12:45.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.564+0000 7f815249a700 1 Processor -- start 2023-11-28T11:12:45.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.564+0000 7f815249a700 1 -- start start 2023-11-28T11:12:45.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f815249a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f814c071340 0x7f814c0c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f815249a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c0ce140 0x7f814c15b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f815249a700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f814c15c710 0x7f814c14dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f815249a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15bc80 con 0x7f814c0ce140 2023-11-28T11:12:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f815249a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15acc0 con 0x7f814c071340 2023-11-28T11:12:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f815249a700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0ad8f0 con 0x7f814c15c710 2023-11-28T11:12:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c0ce140 0x7f814c15b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c0ce140 0x7f814c15b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53740/0 (socket says 172.21.15.83:53740) 2023-11-28T11:12:45.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 -- 172.21.15.83:0/1727643096 learned_addr learned my addr 172.21.15.83:0/1727643096 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 -- 172.21.15.83:0/1727643096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f814c15c710 msgr2=0x7f814c14dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f814c15c710 0x7f814c14dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814bfff700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f814c071340 0x7f814c0c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 -- 172.21.15.83:0/1727643096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f814c071340 msgr2=0x7f814c0c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f814c071340 0x7f814c0c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.565+0000 7f814b7fe700 1 -- 172.21.15.83:0/1727643096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140009550 con 0x7f814c0ce140 2023-11-28T11:12:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.566+0000 7f814b7fe700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c0ce140 0x7f814c15b740 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8140009130 tx=0x7f8140004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.566+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140011630 con 0x7f814c0ce140 2023-11-28T11:12:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.566+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8140011790 con 0x7f814c0ce140 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.566+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c0c9f10 con 0x7f814c0ce140 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.566+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140011b20 con 0x7f814c0ce140 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.566+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c0cd410 con 0x7f814c0ce140 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.567+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f814c15cb80 con 0x7f814c0ce140 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.567+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f814002a070 con 0x7f814c0ce140 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.567+0000 7f8148ff9700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81380682f0 0x7f813806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8140086f30 con 0x7f814c0ce140 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f814bfff700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81380682f0 0x7f813806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f8148ff9700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7f813806eb90 0x7f8138070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 --> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81380716b0 con 0x7f813806eb90 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81400895e0 con 0x7f814c0ce140 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f8150a37700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7f813806eb90 0x7f8138070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f814bfff700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81380682f0 0x7f813806a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f813c009950 tx=0x7f813c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.568+0000 7f8150a37700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7f813806eb90 0x7f8138070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.569+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== osd.2 v2:172.21.15.83:6824/103544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f81380716b0 con 0x7f813806eb90 2023-11-28T11:12:45.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 --> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f814c0742b0 con 0x7f813806eb90 2023-11-28T11:12:45.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f8148ff9700 1 -- 172.21.15.83:0/1727643096 <== osd.2 v2:172.21.15.83:6824/103544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f814c0742b0 con 0x7f813806eb90 2023-11-28T11:12:45.583 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:45.584 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7f813806eb90 msgr2=0x7f8138070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6824/103544,v1:172.21.15.83:6825/103544] conn(0x7f813806eb90 0x7f8138070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81380682f0 msgr2=0x7f813806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f81380682f0 0x7f813806a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f813c009950 tx=0x7f813c008040 comp rx=0 tx=0).stop 2023-11-28T11:12:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c0ce140 msgr2=0x7f814c15b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 --2- 172.21.15.83:0/1727643096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f814c0ce140 0x7f814c15b740 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8140009130 tx=0x7f8140004750 comp rx=0 tx=0).stop 2023-11-28T11:12:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f8150a37700 1 -- 172.21.15.83:0/1727643096 reap_dead start 2023-11-28T11:12:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 shutdown_connections 2023-11-28T11:12:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 >> 172.21.15.83:0/1727643096 conn(0x7f814c0cf300 msgr2=0x7f814c1590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 shutdown_connections 2023-11-28T11:12:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.583+0000 7f815249a700 1 -- 172.21.15.83:0/1727643096 wait complete. 2023-11-28T11:12:45.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T11:12:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.716+0000 7efc0e6ab700 1 Processor -- start 2023-11-28T11:12:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.716+0000 7efc0e6ab700 1 -- start start 2023-11-28T11:12:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc0e6ab700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc080cc020 0x7efc08160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc0e6ab700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7efc080cea20 0x7efc080cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc0e6ab700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc080ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc0e6ab700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc080cbc70 con 0x7efc080cc020 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc0e6ab700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc08157120 con 0x7efc080cea20 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc0e6ab700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0815d040 con 0x7efc0815d4d0 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc07fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc080cc020 0x7efc08160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc07fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc080cc020 0x7efc08160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53748/0 (socket says 172.21.15.83:53748) 2023-11-28T11:12:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.717+0000 7efc07fff700 1 -- 172.21.15.83:0/3432426732 learned_addr learned my addr 172.21.15.83:0/3432426732 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc0cc48700 1 --2- 172.21.15.83:0/3432426732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc080ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc077fe700 1 --2- 172.21.15.83:0/3432426732 >> v2:172.21.15.90:3300/0 conn(0x7efc080cea20 0x7efc080cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc07fff700 1 -- 172.21.15.83:0/3432426732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 msgr2=0x7efc080ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc07fff700 1 --2- 172.21.15.83:0/3432426732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc080ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc07fff700 1 -- 172.21.15.83:0/3432426732 >> v2:172.21.15.90:3300/0 conn(0x7efc080cea20 msgr2=0x7efc080cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc07fff700 1 --2- 172.21.15.83:0/3432426732 >> v2:172.21.15.90:3300/0 conn(0x7efc080cea20 0x7efc080cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc07fff700 1 -- 172.21.15.83:0/3432426732 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc080ca3c0 con 0x7efc080cc020 2023-11-28T11:12:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc07fff700 1 --2- 172.21.15.83:0/3432426732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc08160270 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efc0806e440 tx=0x7efbf0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf264be359ea2b86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc06ffd700 1 -- 172.21.15.83:0/3432426732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000b870 con 0x7efc080cc020 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc06ffd700 1 -- 172.21.15.83:0/3432426732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf000b9d0 con 0x7efc080cc020 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.718+0000 7efc06ffd700 1 -- 172.21.15.83:0/3432426732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000bd60 con 0x7efc080cc020 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 -- 172.21.15.83:0/3432426732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 msgr2=0x7efc08160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/3432426732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc08160270 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efc0806e440 tx=0x7efbf0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 -- 172.21.15.83:0/3432426732 shutdown_connections 2023-11-28T11:12:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/3432426732 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc080ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/3432426732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc08160270 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/3432426732 >> v2:172.21.15.90:3300/0 conn(0x7efc080cea20 0x7efc080cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 -- 172.21.15.83:0/3432426732 >> 172.21.15.83:0/3432426732 conn(0x7efc080b1090 msgr2=0x7efc080b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 -- 172.21.15.83:0/3432426732 shutdown_connections 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.719+0000 7efc0e6ab700 1 -- 172.21.15.83:0/3432426732 wait complete. 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.720+0000 7efc0e6ab700 1 Processor -- start 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.720+0000 7efc0e6ab700 1 -- start start 2023-11-28T11:12:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0e6ab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc081527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0e6ab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc080cea20 0x7efc08152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0e6ab700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc081e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0e6ab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08152640 con 0x7efc080cc020 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0e6ab700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc080c65d0 con 0x7efc080cea20 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0e6ab700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc080c68b0 con 0x7efc0815d4d0 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc081527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc081527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53754/0 (socket says 172.21.15.83:53754) 2023-11-28T11:12:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 -- 172.21.15.83:0/1500795562 learned_addr learned my addr 172.21.15.83:0/1500795562 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc077fe700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc080cea20 0x7efc08152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc0cc48700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc081e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 -- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 msgr2=0x7efc081e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc081e8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 -- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc080cea20 msgr2=0x7efc08152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc080cea20 0x7efc08152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 -- 172.21.15.83:0/1500795562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf0009550 con 0x7efc080cc020 2023-11-28T11:12:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc07fff700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc081527b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efbf0000c00 tx=0x7efbf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.721+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0004310 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.722+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf001e070 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.722+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc080cc930 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.722+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00036a0 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.722+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc081607b0 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.723+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc08160a50 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.723+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efbf0029020 con 0x7efc080cc020 2023-11-28T11:12:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.723+0000 7efc04ff9700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efbe80682f0 0x7efbe806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.723+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7efbf0086c50 con 0x7efc080cc020 2023-11-28T11:12:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.723+0000 7efc077fe700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efbe80682f0 0x7efbe806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.724+0000 7efc077fe700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efbe80682f0 0x7efbe806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc081e7580 tx=0x7efbf8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.727+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efbf00535a0 con 0x7efc080cc020 2023-11-28T11:12:45.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.910+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7efc080742b0 con 0x7efbe80682f0 2023-11-28T11:12:45.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.910+0000 7efc04ff9700 1 -- 172.21.15.83:0/1500795562 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efc080742b0 con 0x7efbe80682f0 2023-11-28T11:12:45.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efbe80682f0 msgr2=0x7efbe806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efbe80682f0 0x7efbe806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc081e7580 tx=0x7efbf8008040 comp rx=0 tx=0).stop 2023-11-28T11:12:45.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 msgr2=0x7efc081527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:45.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc081527b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efbf0000c00 tx=0x7efbf0004750 comp rx=0 tx=0).stop 2023-11-28T11:12:45.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 shutdown_connections 2023-11-28T11:12:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7efc0815d4d0 0x7efc081e8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7efbe80682f0 0x7efbe806a7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc080cc020 0x7efc081527b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 --2- 172.21.15.83:0/1500795562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc080cea20 0x7efc08152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 >> 172.21.15.83:0/1500795562 conn(0x7efc080b1090 msgr2=0x7efc0815d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 shutdown_connections 2023-11-28T11:12:45.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:45.912+0000 7efc0e6ab700 1 -- 172.21.15.83:0/1500795562 wait complete. 2023-11-28T11:12:45.915 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T11:12:45.925 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T11:12:45.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:46.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.048+0000 7f947f7a0700 1 Processor -- start 2023-11-28T11:12:46.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.048+0000 7f947f7a0700 1 -- start start 2023-11-28T11:12:46.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947f7a0700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f94780cd1a0 0x7f94780b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947f7a0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94780b56f0 0x7f94780cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947f7a0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780b5b30 con 0x7f947815a930 2023-11-28T11:12:46.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947f7a0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947806bad0 con 0x7f94780b56f0 2023-11-28T11:12:46.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947f7a0700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947806bdb0 con 0x7f94780cd1a0 2023-11-28T11:12:46.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947dd3d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94780b56f0 0x7f94780cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947dd3d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94780b56f0 0x7f94780cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53766/0 (socket says 172.21.15.83:53766) 2023-11-28T11:12:46.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947dd3d700 1 -- 172.21.15.83:0/2920991474 learned_addr learned my addr 172.21.15.83:0/2920991474 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:46.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947d53c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f947815a930 0x7f947806b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51622/0 (socket says 172.21.15.83:51622) 2023-11-28T11:12:46.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947cd3b700 1 --2- 172.21.15.83:0/2920991474 >> v2:172.21.15.90:3300/0 conn(0x7f94780cd1a0 0x7f94780b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947dd3d700 1 -- 172.21.15.83:0/2920991474 >> v2:172.21.15.90:3300/0 conn(0x7f94780cd1a0 msgr2=0x7f94780b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947dd3d700 1 --2- 172.21.15.83:0/2920991474 >> v2:172.21.15.90:3300/0 conn(0x7f94780cd1a0 0x7f94780b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.049+0000 7f947dd3d700 1 -- 172.21.15.83:0/2920991474 >> v1:172.21.15.83:6789/0 conn(0x7f947815a930 legacy=0x7f947806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947dd3d700 1 -- 172.21.15.83:0/2920991474 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780ca390 con 0x7f94780b56f0 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f946ffff700 1 -- 172.21.15.83:0/2920991474 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639666682 0 0) 0x7f94780b5b30 con 0x7f947815a930 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947dd3d700 1 --2- 172.21.15.83:0/2920991474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94780cab90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f94780af640 tx=0x7f947000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a641088958df1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f946ffff700 1 -- 172.21.15.83:0/2920991474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000e070 con 0x7f94780b56f0 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f946ffff700 1 -- 172.21.15.83:0/2920991474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947000b9c0 con 0x7f94780b56f0 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f946ffff700 1 -- 172.21.15.83:0/2920991474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000bd70 con 0x7f94780b56f0 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 -- 172.21.15.83:0/2920991474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 msgr2=0x7f94780cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 --2- 172.21.15.83:0/2920991474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94780cab90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f94780af640 tx=0x7f947000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:46.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 -- 172.21.15.83:0/2920991474 shutdown_connections 2023-11-28T11:12:46.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 --2- 172.21.15.83:0/2920991474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94780cab90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 --2- 172.21.15.83:0/2920991474 >> v2:172.21.15.90:3300/0 conn(0x7f94780cd1a0 0x7f94780b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 -- 172.21.15.83:0/2920991474 >> 172.21.15.83:0/2920991474 conn(0x7f94780cc690 msgr2=0x7f94780cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 -- 172.21.15.83:0/2920991474 shutdown_connections 2023-11-28T11:12:46.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.050+0000 7f947f7a0700 1 -- 172.21.15.83:0/2920991474 wait complete. 2023-11-28T11:12:46.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 Processor -- start 2023-11-28T11:12:46.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 -- start start 2023-11-28T11:12:46.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94781521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94780cd1a0 0x7f94780c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f947815a930 0x7f94780c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781f3510 con 0x7f94780b56f0 2023-11-28T11:12:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780c6660 con 0x7f947815a930 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947f7a0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780c5dc0 con 0x7f94780cd1a0 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947d53c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94781521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947d53c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94781521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53768/0 (socket says 172.21.15.83:53768) 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.052+0000 7f947d53c700 1 -- 172.21.15.83:0/3283623797 learned_addr learned my addr 172.21.15.83:0/3283623797 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947d53c700 1 -- 172.21.15.83:0/3283623797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94780cd1a0 msgr2=0x7f94780c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947d53c700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94780cd1a0 0x7f94780c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947d53c700 1 -- 172.21.15.83:0/3283623797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f947815a930 msgr2=0x7f94780c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947d53c700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f947815a930 0x7f94780c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947d53c700 1 -- 172.21.15.83:0/3283623797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470009b30 con 0x7f94780b56f0 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947d53c700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94781521d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f946800c550 tx=0x7f946800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468004c80 con 0x7f94780b56f0 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468004e00 con 0x7f94780b56f0 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947815ad00 con 0x7f94780b56f0 2023-11-28T11:12:46.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468007310 con 0x7f94780b56f0 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.053+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94780ca670 con 0x7f94780b56f0 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.054+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f947806c090 con 0x7f94780b56f0 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.054+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f946801a020 con 0x7f94780b56f0 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f946dffb700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f945c0682f0 0x7f945c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9468085720 con 0x7f94780b56f0 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f947cd3b700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f945c0682f0 0x7f945c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f946dffb700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7f945c06eb90 0x7f945c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 --> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f945c0716b0 con 0x7f945c06eb90 2023-11-28T11:12:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9468087dd0 con 0x7f94780b56f0 2023-11-28T11:12:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f947dd3d700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7f945c06eb90 0x7f945c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f947cd3b700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f945c0682f0 0x7f945c06a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f947400a8d0 tx=0x7f9474008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.055+0000 7f947dd3d700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7f945c06eb90 0x7f945c070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.056+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== osd.3 v2:172.21.15.83:6808/103549 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f945c0716b0 con 0x7f945c06eb90 2023-11-28T11:12:46.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.070+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 --> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f94781f3650 con 0x7f945c06eb90 2023-11-28T11:12:46.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f946dffb700 1 -- 172.21.15.83:0/3283623797 <== osd.3 v2:172.21.15.83:6808/103549 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f947000bb30 con 0x7f945c06eb90 2023-11-28T11:12:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:46.072 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:46.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7f945c06eb90 msgr2=0x7f945c070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6808/103549,v1:172.21.15.83:6810/103549] conn(0x7f945c06eb90 0x7f945c070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f945c0682f0 msgr2=0x7f945c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f945c0682f0 0x7f945c06a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f947400a8d0 tx=0x7f9474008040 comp rx=0 tx=0).stop 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 msgr2=0x7f94781521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 --2- 172.21.15.83:0/3283623797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94780b56f0 0x7f94781521d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f946800c550 tx=0x7f946800c910 comp rx=0 tx=0).stop 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947dd3d700 1 -- 172.21.15.83:0/3283623797 reap_dead start 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 shutdown_connections 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 >> 172.21.15.83:0/3283623797 conn(0x7f94780cc690 msgr2=0x7f94780cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 shutdown_connections 2023-11-28T11:12:46.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.071+0000 7f947f7a0700 1 -- 172.21.15.83:0/3283623797 wait complete. 2023-11-28T11:12:46.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T11:12:46.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.203+0000 7feb68439700 1 Processor -- start 2023-11-28T11:12:46.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.203+0000 7feb68439700 1 -- start start 2023-11-28T11:12:46.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb68439700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb600d0ae0 0x7feb600d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb68439700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb60151a30 con 0x7feb600ce1b0 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb68439700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb600d1440 con 0x7feb6015e5c0 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb68439700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6015bac0 con 0x7feb600d0ae0 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb669d6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb600d0ae0 0x7feb600d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb659d4700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feb600ce1b0 0x7feb6015b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51638/0 (socket says 172.21.15.83:51638) 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb669d6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb600d0ae0 0x7feb600d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53780/0 (socket says 172.21.15.83:53780) 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.204+0000 7feb659d4700 1 -- 172.21.15.83:0/1891673958 learned_addr learned my addr 172.21.15.83:0/1891673958 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb669d6700 1 -- 172.21.15.83:0/1891673958 >> v1:172.21.15.90:6789/0 conn(0x7feb6015e5c0 legacy=0x7feb6015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb669d6700 1 -- 172.21.15.83:0/1891673958 >> v1:172.21.15.83:6789/0 conn(0x7feb600ce1b0 legacy=0x7feb6015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb669d6700 1 -- 172.21.15.83:0/1891673958 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600cb460 con 0x7feb600d0ae0 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb651d3700 1 -- 172.21.15.83:0/1891673958 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863483484 0 0) 0x7feb60151a30 con 0x7feb600ce1b0 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb669d6700 1 --2- 172.21.15.83:0/1891673958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb600d0ae0 0x7feb600d0f00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feb58009d90 tx=0x7feb5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36966dfec1c605 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb651d3700 1 -- 172.21.15.83:0/1891673958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5800e070 con 0x7feb600d0ae0 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb651d3700 1 -- 172.21.15.83:0/1891673958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5800bb90 con 0x7feb600d0ae0 2023-11-28T11:12:46.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb651d3700 1 -- 172.21.15.83:0/1891673958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58003a20 con 0x7feb600d0ae0 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.205+0000 7feb68439700 1 -- 172.21.15.83:0/1891673958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb600d0ae0 msgr2=0x7feb600d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.206+0000 7feb68439700 1 --2- 172.21.15.83:0/1891673958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb600d0ae0 0x7feb600d0f00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feb58009d90 tx=0x7feb5800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.206+0000 7feb68439700 1 -- 172.21.15.83:0/1891673958 shutdown_connections 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.206+0000 7feb68439700 1 --2- 172.21.15.83:0/1891673958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb600d0ae0 0x7feb600d0f00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.206+0000 7feb68439700 1 -- 172.21.15.83:0/1891673958 >> 172.21.15.83:0/1891673958 conn(0x7feb6006ea50 msgr2=0x7feb60070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.206+0000 7feb68439700 1 -- 172.21.15.83:0/1891673958 shutdown_connections 2023-11-28T11:12:46.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.206+0000 7feb68439700 1 -- 172.21.15.83:0/1891673958 wait complete. 2023-11-28T11:12:46.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.207+0000 7feb68439700 1 Processor -- start 2023-11-28T11:12:46.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.207+0000 7feb68439700 1 -- start start 2023-11-28T11:12:46.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb600d0ae0 0x7feb600cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 0x7feb60150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb600c6b70 0x7feb600ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600d0750 con 0x7feb6015e5c0 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600b6150 con 0x7feb600c6b70 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600b65e0 con 0x7feb600d0ae0 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 0x7feb60150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 0x7feb60150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53784/0 (socket says 172.21.15.83:53784) 2023-11-28T11:12:46.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 -- 172.21.15.83:0/452872550 learned_addr learned my addr 172.21.15.83:0/452872550 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb669d6700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb600c6b70 0x7feb600ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb661d5700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb600d0ae0 0x7feb600cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 -- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb600d0ae0 msgr2=0x7feb600cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb600d0ae0 0x7feb600cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 -- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb600c6b70 msgr2=0x7feb600ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb600c6b70 0x7feb600ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 -- 172.21.15.83:0/452872550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58009a70 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb659d4700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 0x7feb60150300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c5b0 tx=0x7feb5c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c004c80 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c004de0 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6015abb0 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c007310 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.208+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb600cb740 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.209+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6015ae90 con 0x7feb6015e5c0 2023-11-28T11:12:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.210+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb5c01a020 con 0x7feb6015e5c0 2023-11-28T11:12:46.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.210+0000 7feb56ffd700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb440682f0 0x7feb4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.210+0000 7feb661d5700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb440682f0 0x7feb4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.210+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7feb5c085630 con 0x7feb6015e5c0 2023-11-28T11:12:46.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.211+0000 7feb661d5700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb440682f0 0x7feb4406a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feb50004f30 tx=0x7feb50009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.214+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb5c052050 con 0x7feb6015e5c0 2023-11-28T11:12:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.397+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7feb600742b0 con 0x7feb440682f0 2023-11-28T11:12:46.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.397+0000 7feb56ffd700 1 -- 172.21.15.83:0/452872550 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7feb600742b0 con 0x7feb440682f0 2023-11-28T11:12:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb440682f0 msgr2=0x7feb4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb440682f0 0x7feb4406a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feb50004f30 tx=0x7feb50009040 comp rx=0 tx=0).stop 2023-11-28T11:12:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 msgr2=0x7feb60150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 0x7feb60150300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c5b0 tx=0x7feb5c00c970 comp rx=0 tx=0).stop 2023-11-28T11:12:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 shutdown_connections 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb600c6b70 0x7feb600ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb600d0ae0 0x7feb600cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7feb440682f0 0x7feb4406a7a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 --2- 172.21.15.83:0/452872550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6015e5c0 0x7feb60150300 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 >> 172.21.15.83:0/452872550 conn(0x7feb600ce1b0 msgr2=0x7feb601511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 shutdown_connections 2023-11-28T11:12:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.399+0000 7feb68439700 1 -- 172.21.15.83:0/452872550 wait complete. 2023-11-28T11:12:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T11:12:46.412 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T11:12:46.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:46.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 Processor -- start 2023-11-28T11:12:46.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 -- start start 2023-11-28T11:12:46.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa25015cd30 0x7fa2500cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 0x7fa2500cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25015d100 con 0x7fa2500b17e0 2023-11-28T11:12:46.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2500b2330 con 0x7fa25015cd30 2023-11-28T11:12:46.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.536+0000 7fa257245700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25006beb0 con 0x7fa25006cf30 2023-11-28T11:12:46.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa2557e2700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa2500b17e0 0x7fa2500b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51654/0 (socket says 172.21.15.83:51654) 2023-11-28T11:12:46.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa2557e2700 1 -- 172.21.15.83:0/1121179590 learned_addr learned my addr 172.21.15.83:0/1121179590 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa254fe1700 1 --2- 172.21.15.83:0/1121179590 >> v2:172.21.15.90:3300/0 conn(0x7fa25015cd30 0x7fa2500cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24ffff700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 0x7fa2500cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955167502 0 0) 0x7fa248001490 con 0x7fa2500b17e0 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2400035b0 con 0x7fa2500b17e0 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078206269 0 0) 0x7fa2400035b0 con 0x7fa2500b17e0 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 msgr2=0x7fa2500cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 0x7fa2500cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 >> v2:172.21.15.90:3300/0 conn(0x7fa25015cd30 msgr2=0x7fa2500cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 --2- 172.21.15.83:0/1121179590 >> v2:172.21.15.90:3300/0 conn(0x7fa25015cd30 0x7fa2500cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25006c170 con 0x7fa2500b17e0 2023-11-28T11:12:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fa248001b80 con 0x7fa2500b17e0 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 >> v1:172.21.15.83:6789/0 conn(0x7fa2500b17e0 legacy=0x7fa2500b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 0x7fa240003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa240004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240004d90 0x7fa240005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500b2330 con 0x7fa240003f80 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25006beb0 con 0x7fa240004d90 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24000ba10 con 0x7fa25006cf30 2023-11-28T11:12:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa24ffff700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa240004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa254fe1700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 0x7fa240003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.537+0000 7fa2557e2700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240004d90 0x7fa240005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24ffff700 1 -- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 msgr2=0x7fa240003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24ffff700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa25006cf30 0x7fa240003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24ffff700 1 -- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240004d90 msgr2=0x7fa240005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24ffff700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240004d90 0x7fa240005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24ffff700 1 -- 172.21.15.83:0/1121179590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2400035b0 con 0x7fa240003f80 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24ffff700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa240004850 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009b30 tx=0x7fa240005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c881f9d07af6e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00e6c0 con 0x7fa240003f80 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23c00e820 con 0x7fa240003f80 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.538+0000 7fa24f7fe700 1 -- 172.21.15.83:0/1121179590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00eb90 con 0x7fa240003f80 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa257245700 1 -- 172.21.15.83:0/1121179590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 msgr2=0x7fa240004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa257245700 1 --2- 172.21.15.83:0/1121179590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa240004850 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009b30 tx=0x7fa240005880 comp rx=0 tx=0).stop 2023-11-28T11:12:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa2557e2700 1 -- 172.21.15.83:0/1121179590 reap_dead start 2023-11-28T11:12:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa257245700 1 -- 172.21.15.83:0/1121179590 shutdown_connections 2023-11-28T11:12:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa257245700 1 -- 172.21.15.83:0/1121179590 >> 172.21.15.83:0/1121179590 conn(0x7fa2500cc690 msgr2=0x7fa2500cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa257245700 1 -- 172.21.15.83:0/1121179590 shutdown_connections 2023-11-28T11:12:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.539+0000 7fa257245700 1 -- 172.21.15.83:0/1121179590 wait complete. 2023-11-28T11:12:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.540+0000 7fa257245700 1 Processor -- start 2023-11-28T11:12:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.540+0000 7fa257245700 1 -- start start 2023-11-28T11:12:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa2500b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2500b3470 0x7fa25015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa250148b80 0x7fa250149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500b44d0 con 0x7fa240003f80 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500c7760 con 0x7fa250148b80 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa250149560 con 0x7fa2500b3470 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa2500b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa2500b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53802/0 (socket says 172.21.15.83:53802) 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 -- 172.21.15.83:0/3899186041 learned_addr learned my addr 172.21.15.83:0/3899186041 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa24ffff700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2500b3470 0x7fa25015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa2557e2700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa250148b80 0x7fa250149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 -- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2500b3470 msgr2=0x7fa25015ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2500b3470 0x7fa25015ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 -- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa250148b80 msgr2=0x7fa250149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa250148b80 0x7fa250149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 -- 172.21.15.83:0/3899186041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25006c730 con 0x7fa240003f80 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa2557e2700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa250148b80 0x7fa250149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa254fe1700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa2500b4fa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa244011aa0 tx=0x7fa244011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400f940 con 0x7fa240003f80 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24400faa0 con 0x7fa240003f80 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25006c450 con 0x7fa240003f80 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400fe50 con 0x7fa240003f80 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.541+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25006ca10 con 0x7fa240003f80 2023-11-28T11:12:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.542+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2500742b0 con 0x7fa240003f80 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa244014440 con 0x7fa240003f80 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24d7fa700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa2340682f0 0x7fa23406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa244086470 con 0x7fa240003f80 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24ffff700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa2340682f0 0x7fa23406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24d7fa700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fa23406eb90 0x7fa234070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 --> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2340716b0 con 0x7fa23406eb90 2023-11-28T11:12:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.543+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa244088b20 con 0x7fa240003f80 2023-11-28T11:12:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.544+0000 7fa2557e2700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fa23406eb90 0x7fa234070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.544+0000 7fa24ffff700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa2340682f0 0x7fa23406a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0090b0 tx=0x7fa23c01f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.544+0000 7fa2557e2700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fa23406eb90 0x7fa234070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.545+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== osd.4 v2:172.21.15.90:6816/103766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa2340716b0 con 0x7fa23406eb90 2023-11-28T11:12:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.559+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 --> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa2500b1cb0 con 0x7fa23406eb90 2023-11-28T11:12:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.559+0000 7fa24d7fa700 1 -- 172.21.15.83:0/3899186041 <== osd.4 v2:172.21.15.90:6816/103766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa2500b1cb0 con 0x7fa23406eb90 2023-11-28T11:12:46.560 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:46.560 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fa23406eb90 msgr2=0x7fa234070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.90:6816/103766,v1:172.21.15.90:6817/103766] conn(0x7fa23406eb90 0x7fa234070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa2340682f0 msgr2=0x7fa23406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fa2340682f0 0x7fa23406a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0090b0 tx=0x7fa23c01f040 comp rx=0 tx=0).stop 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 msgr2=0x7fa2500b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 --2- 172.21.15.83:0/3899186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa240003f80 0x7fa2500b4fa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa244011aa0 tx=0x7fa244011db0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa2557e2700 1 -- 172.21.15.83:0/3899186041 reap_dead start 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 shutdown_connections 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 >> 172.21.15.83:0/3899186041 conn(0x7fa2500cc690 msgr2=0x7fa2500af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 shutdown_connections 2023-11-28T11:12:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.560+0000 7fa257245700 1 -- 172.21.15.83:0/3899186041 wait complete. 2023-11-28T11:12:46.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T11:12:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.694+0000 7f2697d84700 1 Processor -- start 2023-11-28T11:12:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.694+0000 7f2697d84700 1 -- start start 2023-11-28T11:12:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2697d84700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f269015e5c0 0x7f269015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2697d84700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015b850 0x7f26900ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2697d84700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2690151a30 con 0x7f269015b3b0 2023-11-28T11:12:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2697d84700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26900cb460 con 0x7f269015e5c0 2023-11-28T11:12:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2697d84700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900cb740 con 0x7f269015b850 2023-11-28T11:12:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2695b20700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f269015e5c0 0x7f269015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2695b20700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f269015e5c0 0x7f269015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53812/0 (socket says 172.21.15.83:53812) 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f269531f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f269015b3b0 0x7f26900d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51672/0 (socket says 172.21.15.83:51672) 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2695b20700 1 -- 172.21.15.83:0/2042514811 learned_addr learned my addr 172.21.15.83:0/2042514811 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.695+0000 7f2696321700 1 --2- 172.21.15.83:0/2042514811 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015b850 0x7f26900ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2695b20700 1 -- 172.21.15.83:0/2042514811 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015b850 msgr2=0x7f26900ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2695b20700 1 --2- 172.21.15.83:0/2042514811 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015b850 0x7f26900ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2694b1e700 1 -- 172.21.15.83:0/2042514811 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920594941 0 0) 0x7f2690151a30 con 0x7f269015b3b0 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2695b20700 1 -- 172.21.15.83:0/2042514811 >> v1:172.21.15.83:6789/0 conn(0x7f269015b3b0 legacy=0x7f26900d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2695b20700 1 -- 172.21.15.83:0/2042514811 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269015abb0 con 0x7f269015e5c0 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2695b20700 1 --2- 172.21.15.83:0/2042514811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015e5c0 0x7f269015e990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f268c0098f0 tx=0x7f268c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322e994bd9eaeabe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2694b1e700 1 -- 172.21.15.83:0/2042514811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00b8a0 con 0x7f269015e5c0 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2694b1e700 1 -- 172.21.15.83:0/2042514811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c00ba00 con 0x7f269015e5c0 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2694b1e700 1 -- 172.21.15.83:0/2042514811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00bd90 con 0x7f269015e5c0 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2697d84700 1 -- 172.21.15.83:0/2042514811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015e5c0 msgr2=0x7f269015e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2697d84700 1 --2- 172.21.15.83:0/2042514811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015e5c0 0x7f269015e990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f268c0098f0 tx=0x7f268c009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2697d84700 1 -- 172.21.15.83:0/2042514811 shutdown_connections 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2697d84700 1 --2- 172.21.15.83:0/2042514811 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015b850 0x7f26900ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2697d84700 1 --2- 172.21.15.83:0/2042514811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015e5c0 0x7f269015e990 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.696+0000 7f2697d84700 1 -- 172.21.15.83:0/2042514811 >> 172.21.15.83:0/2042514811 conn(0x7f269006ea50 msgr2=0x7f2690070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.697+0000 7f2697d84700 1 -- 172.21.15.83:0/2042514811 shutdown_connections 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.697+0000 7f2697d84700 1 -- 172.21.15.83:0/2042514811 wait complete. 2023-11-28T11:12:46.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 Processor -- start 2023-11-28T11:12:46.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 -- start start 2023-11-28T11:12:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 0x7f2690149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015e5c0 0x7f2690153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26901539d0 0x7f26900b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26900c5590 con 0x7f269015b850 2023-11-28T11:12:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690153f50 con 0x7f26901539d0 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.698+0000 7f2697d84700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2690154230 con 0x7f269015e5c0 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 0x7f2690149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 0x7f2690149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53818/0 (socket says 172.21.15.83:53818) 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 -- 172.21.15.83:0/3263536419 learned_addr learned my addr 172.21.15.83:0/3263536419 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 -- 172.21.15.83:0/3263536419 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015e5c0 msgr2=0x7f2690153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015e5c0 0x7f2690153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 -- 172.21.15.83:0/3263536419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26901539d0 msgr2=0x7f26900b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26901539d0 0x7f26900b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 -- 172.21.15.83:0/3263536419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c009550 con 0x7f269015b850 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2695b20700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 0x7f2690149050 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f268c009c40 tx=0x7f268c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0036a0 con 0x7f269015b850 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c003800 con 0x7f269015b850 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269015eed0 con 0x7f269015b850 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c003b70 con 0x7f269015b850 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.699+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269015ae90 con 0x7f269015b850 2023-11-28T11:12:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.700+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26900cba20 con 0x7f269015b850 2023-11-28T11:12:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.700+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f268c02d070 con 0x7f269015b850 2023-11-28T11:12:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.701+0000 7f26867fc700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f26740682f0 0x7f267406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.701+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f268c021030 con 0x7f269015b850 2023-11-28T11:12:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.701+0000 7f269531f700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f26740682f0 0x7f267406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.701+0000 7f269531f700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f26740682f0 0x7f267406a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f26900b6800 tx=0x7f26800079f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.705+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f268c053d10 con 0x7f269015b850 2023-11-28T11:12:46.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.887+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f26900742b0 con 0x7f26740682f0 2023-11-28T11:12:46.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.887+0000 7f26867fc700 1 -- 172.21.15.83:0/3263536419 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f26900742b0 con 0x7f26740682f0 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f26740682f0 msgr2=0x7f267406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f26740682f0 0x7f267406a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f26900b6800 tx=0x7f26800079f0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 msgr2=0x7f2690149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 0x7f2690149050 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f268c009c40 tx=0x7f268c004750 comp rx=0 tx=0).stop 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 shutdown_connections 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26901539d0 0x7f26900b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f26740682f0 0x7f267406a7a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269015b850 0x7f2690149050 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 --2- 172.21.15.83:0/3263536419 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f269015e5c0 0x7f2690153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:46.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 >> 172.21.15.83:0/3263536419 conn(0x7f269015b3b0 msgr2=0x7f26900d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:46.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 shutdown_connections 2023-11-28T11:12:46.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:46.889+0000 7f2697d84700 1 -- 172.21.15.83:0/3263536419 wait complete. 2023-11-28T11:12:46.891 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T11:12:46.902 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T11:12:46.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:47.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.025+0000 7f752d213700 1 Processor -- start 2023-11-28T11:12:47.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.025+0000 7f752d213700 1 -- start start 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752d213700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f75280cb520 0x7f75280b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752d213700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f752806cf30 0x7f75280b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752d213700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752815d100 con 0x7f752815cd30 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752d213700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752806beb0 con 0x7f752806cf30 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752d213700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752806c170 con 0x7f75280cb520 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f752806cf30 0x7f75280b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f752806cf30 0x7f75280b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53830/0 (socket says 172.21.15.83:53830) 2023-11-28T11:12:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 -- 172.21.15.83:0/2282752213 learned_addr learned my addr 172.21.15.83:0/2282752213 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752659c700 1 --2- 172.21.15.83:0/2282752213 >> v2:172.21.15.90:3300/0 conn(0x7f75280cb520 0x7f75280b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 -- 172.21.15.83:0/2282752213 >> v2:172.21.15.90:3300/0 conn(0x7f75280cb520 msgr2=0x7f75280b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 --2- 172.21.15.83:0/2282752213 >> v2:172.21.15.90:3300/0 conn(0x7f75280cb520 0x7f75280b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 -- 172.21.15.83:0/2282752213 >> v1:172.21.15.90:6789/0 conn(0x7f752815cd30 legacy=0x7f75280cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 -- 172.21.15.83:0/2282752213 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752806c450 con 0x7f752806cf30 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.026+0000 7f752759e700 1 --2- 172.21.15.83:0/2282752213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752806cf30 0x7f75280b1d20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7528160700 tx=0x7f752000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86607f9cb9344e1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f7525d9b700 1 -- 172.21.15.83:0/2282752213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000e070 con 0x7f752806cf30 2023-11-28T11:12:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f7525d9b700 1 -- 172.21.15.83:0/2282752213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7520003a20 con 0x7f752806cf30 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f7525d9b700 1 -- 172.21.15.83:0/2282752213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520003dd0 con 0x7f752806cf30 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 -- 172.21.15.83:0/2282752213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752806cf30 msgr2=0x7f75280b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 --2- 172.21.15.83:0/2282752213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752806cf30 0x7f75280b1d20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7528160700 tx=0x7f752000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 -- 172.21.15.83:0/2282752213 shutdown_connections 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 --2- 172.21.15.83:0/2282752213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752806cf30 0x7f75280b1d20 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 --2- 172.21.15.83:0/2282752213 >> v2:172.21.15.90:3300/0 conn(0x7f75280cb520 0x7f75280b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 -- 172.21.15.83:0/2282752213 >> 172.21.15.83:0/2282752213 conn(0x7f75280cc690 msgr2=0x7f75280cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 -- 172.21.15.83:0/2282752213 shutdown_connections 2023-11-28T11:12:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.027+0000 7f752d213700 1 -- 172.21.15.83:0/2282752213 wait complete. 2023-11-28T11:12:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 Processor -- start 2023-11-28T11:12:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 -- start start 2023-11-28T11:12:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752806cf30 0x7f752814ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cb520 0x7f752815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f752815cd30 0x7f75281f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752815ff30 con 0x7f75280cb520 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752815f030 con 0x7f752806cf30 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752d213700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752815f2b0 con 0x7f752815cd30 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cb520 0x7f752815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cb520 0x7f752815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53834/0 (socket says 172.21.15.83:53834) 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.029+0000 7f752659c700 1 -- 172.21.15.83:0/3997880509 learned_addr learned my addr 172.21.15.83:0/3997880509 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f7526d9d700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752806cf30 0x7f752814ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752759e700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f752815cd30 0x7f75281f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752659c700 1 -- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f752815cd30 msgr2=0x7f75281f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752659c700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f752815cd30 0x7f75281f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752659c700 1 -- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752806cf30 msgr2=0x7f752814ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752659c700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752806cf30 0x7f752814ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752659c700 1 -- 172.21.15.83:0/3997880509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7520009b30 con 0x7f75280cb520 2023-11-28T11:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f7526d9d700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752806cf30 0x7f752814ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752659c700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cb520 0x7f752815d4d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f751800e810 tx=0x7f751800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518013610 con 0x7f75280cb520 2023-11-28T11:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518013770 con 0x7f75280cb520 2023-11-28T11:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752806ca10 con 0x7f75280cb520 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518013b20 con 0x7f75280cb520 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.030+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752806c730 con 0x7f75280cb520 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.031+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75280b2260 con 0x7f75280cb520 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.031+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7518023070 con 0x7f75280cb520 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f750f7fe700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75140682f0 0x7f751406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f75180857a0 con 0x7f75280cb520 2023-11-28T11:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f7526d9d700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75140682f0 0x7f751406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f750f7fe700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f751406eb90 0x7f7514070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75140716b0 con 0x7f751406eb90 2023-11-28T11:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7518087e50 con 0x7f75280cb520 2023-11-28T11:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f752759e700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f751406eb90 0x7f7514070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.032+0000 7f7526d9d700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75140682f0 0x7f751406a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f75280cbb10 tx=0x7f751c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.033+0000 7f752759e700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f751406eb90 0x7f7514070fe0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.033+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== osd.5 v2:172.21.15.90:6824/103767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f75140716b0 con 0x7f751406eb90 2023-11-28T11:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.047+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 --> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f75280742b0 con 0x7f751406eb90 2023-11-28T11:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.048+0000 7f750f7fe700 1 -- 172.21.15.83:0/3997880509 <== osd.5 v2:172.21.15.90:6824/103767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f75280742b0 con 0x7f751406eb90 2023-11-28T11:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.048+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f751406eb90 msgr2=0x7f7514070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.048+0000 7f752d213700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.90:6824/103767,v1:172.21.15.90:6825/103767] conn(0x7f751406eb90 0x7f7514070fe0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75140682f0 msgr2=0x7f751406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f75140682f0 0x7f751406a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f75280cbb10 tx=0x7f751c008040 comp rx=0 tx=0).stop 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cb520 msgr2=0x7f752815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 --2- 172.21.15.83:0/3997880509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cb520 0x7f752815d4d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f751800e810 tx=0x7f751800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752759e700 1 -- 172.21.15.83:0/3997880509 reap_dead start 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 shutdown_connections 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 >> 172.21.15.83:0/3997880509 conn(0x7f75280cc690 msgr2=0x7f75280cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 shutdown_connections 2023-11-28T11:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.049+0000 7f752d213700 1 -- 172.21.15.83:0/3997880509 wait complete. 2023-11-28T11:12:47.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T11:12:47.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.182+0000 7fae2eb78700 1 Processor -- start 2023-11-28T11:12:47.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.182+0000 7fae2eb78700 1 -- start start 2023-11-28T11:12:47.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.183+0000 7fae2eb78700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 0x7fae280cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.183+0000 7fae2eb78700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae280c9e70 con 0x7fae28158900 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.183+0000 7fae2eb78700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2815c790 con 0x7fae280cb650 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.183+0000 7fae2eb78700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2815ca70 con 0x7fae28158cd0 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.183+0000 7fae2c914700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fae28158900 0x7fae28159ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51702/0 (socket says 172.21.15.83:51702) 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.183+0000 7fae2c914700 1 -- 172.21.15.83:0/3301708376 learned_addr learned my addr 172.21.15.83:0/3301708376 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258014378 0 0) 0x7fae280c9e70 con 0x7fae28158900 2023-11-28T11:12:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae180035b0 con 0x7fae28158900 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae27fff700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 0x7fae280cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 653271701 0 0) 0x7fae180035b0 con 0x7fae28158900 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 msgr2=0x7fae280cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 0x7fae280cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 >> v1:172.21.15.90:6789/0 conn(0x7fae280cb650 legacy=0x7fae280cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2815cd50 con 0x7fae28158900 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fae1c001670 con 0x7fae28158900 2023-11-28T11:12:47.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 >> v1:172.21.15.83:6789/0 conn(0x7fae28158900 legacy=0x7fae28159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae18003a30 0x7fae18003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 0x7fae18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae18004d90 0x7fae18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2815ca70 con 0x7fae18004d90 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1800bc00 con 0x7fae18003a30 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1800bdc0 con 0x7fae28158cd0 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae18004d90 0x7fae18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2c914700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae18003a30 0x7fae18003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae27fff700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 0x7fae18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 -- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 msgr2=0x7fae18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae28158cd0 0x7fae18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 -- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae18003a30 msgr2=0x7fae18003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae18003a30 0x7fae18003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 -- 172.21.15.83:0/3301708376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180035b0 con 0x7fae18004d90 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.184+0000 7fae2d115700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae18004d90 0x7fae18005200 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c160 tx=0x7fae18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca3e2f56485038f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20001db0 con 0x7fae18004d90 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2000cca0 con 0x7fae18004d90 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae277fe700 1 -- 172.21.15.83:0/3301708376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20003c40 con 0x7fae18004d90 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2eb78700 1 -- 172.21.15.83:0/3301708376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae18004d90 msgr2=0x7fae18005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3301708376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae18004d90 0x7fae18005200 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c160 tx=0x7fae18005880 comp rx=0 tx=0).stop 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2d115700 1 -- 172.21.15.83:0/3301708376 reap_dead start 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2eb78700 1 -- 172.21.15.83:0/3301708376 shutdown_connections 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2eb78700 1 -- 172.21.15.83:0/3301708376 >> 172.21.15.83:0/3301708376 conn(0x7fae280b1090 msgr2=0x7fae280b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2eb78700 1 -- 172.21.15.83:0/3301708376 shutdown_connections 2023-11-28T11:12:47.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.185+0000 7fae2eb78700 1 -- 172.21.15.83:0/3301708376 wait complete. 2023-11-28T11:12:47.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.187+0000 7fae2eb78700 1 Processor -- start 2023-11-28T11:12:47.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.187+0000 7fae2eb78700 1 -- start start 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2eb78700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae2806d3a0 0x7fae280c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2eb78700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae280affa0 0x7fae2806d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2eb78700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 0x7fae28147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2eb78700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae28147580 con 0x7fae2806dcb0 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2eb78700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c2f10 con 0x7fae280affa0 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2eb78700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c31f0 con 0x7fae2806d3a0 2023-11-28T11:12:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 0x7fae28147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 0x7fae28147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53858/0 (socket says 172.21.15.83:53858) 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 -- 172.21.15.83:0/3667661507 learned_addr learned my addr 172.21.15.83:0/3667661507 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2c914700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae2806d3a0 0x7fae280c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae27fff700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae280affa0 0x7fae2806d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 -- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae2806d3a0 msgr2=0x7fae280c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae2806d3a0 0x7fae280c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 -- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae280affa0 msgr2=0x7fae2806d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae280affa0 0x7fae2806d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 -- 172.21.15.83:0/3667661507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2815d310 con 0x7fae2806dcb0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.188+0000 7fae2d115700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 0x7fae28147a10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fae20015040 tx=0x7fae200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.189+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20003e10 con 0x7fae2806dcb0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.189+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2000cb20 con 0x7fae2806dcb0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.189+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2815d030 con 0x7fae2806dcb0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.189+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae200052f0 con 0x7fae2806dcb0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.189+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2806a270 con 0x7fae2806dcb0 2023-11-28T11:12:47.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.190+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae28147730 con 0x7fae2806dcb0 2023-11-28T11:12:47.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.190+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae20029030 con 0x7fae2806dcb0 2023-11-28T11:12:47.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.191+0000 7fae257fa700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae0c0682f0 0x7fae0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.191+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fae20085fa0 con 0x7fae2806dcb0 2023-11-28T11:12:47.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.191+0000 7fae2c914700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae0c0682f0 0x7fae0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.191+0000 7fae2c914700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae0c0682f0 0x7fae0c06a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0010e0 tx=0x7fae1c000ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.195+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae28147730 con 0x7fae2806dcb0 2023-11-28T11:12:47.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.377+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fae28147730 con 0x7fae0c0682f0 2023-11-28T11:12:47.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.377+0000 7fae257fa700 1 -- 172.21.15.83:0/3667661507 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fae28147730 con 0x7fae0c0682f0 2023-11-28T11:12:47.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae0c0682f0 msgr2=0x7fae0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae0c0682f0 0x7fae0c06a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0010e0 tx=0x7fae1c000ef0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 msgr2=0x7fae28147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 0x7fae28147a10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fae20015040 tx=0x7fae200014a0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 shutdown_connections 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fae0c0682f0 0x7fae0c06a7a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2806dcb0 0x7fae28147a10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae2806d3a0 0x7fae280c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 --2- 172.21.15.83:0/3667661507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae280affa0 0x7fae2806d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 >> 172.21.15.83:0/3667661507 conn(0x7fae280b1090 msgr2=0x7fae281e2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 shutdown_connections 2023-11-28T11:12:47.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.379+0000 7fae2eb78700 1 -- 172.21.15.83:0/3667661507 wait complete. 2023-11-28T11:12:47.381 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T11:12:47.391 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T11:12:47.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.515+0000 7f56593b0700 1 Processor -- start 2023-11-28T11:12:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.515+0000 7f56593b0700 1 -- start start 2023-11-28T11:12:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56593b0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f5654167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56593b0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56540b2600 con 0x7f56540d2760 2023-11-28T11:12:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56593b0700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56540b5cf0 con 0x7f56540b2230 2023-11-28T11:12:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56593b0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56540cdb20 con 0x7f56540b5050 2023-11-28T11:12:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56527fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f56540d2760 0x7f56540b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51724/0 (socket says 172.21.15.83:51724) 2023-11-28T11:12:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56527fc700 1 -- 172.21.15.83:0/1537851950 learned_addr learned my addr 172.21.15.83:0/1537851950 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56537fe700 1 --2- 172.21.15.83:0/1537851950 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f5654167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286618985 0 0) 0x7f56540b2600 con 0x7f56540d2760 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f563c0035b0 con 0x7f56540d2760 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304010971 0 0) 0x7f56540b5cf0 con 0x7f56540b2230 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56540b2600 con 0x7f56540b2230 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56537fe700 1 -- 172.21.15.83:0/1537851950 >> v1:172.21.15.90:6789/0 conn(0x7f56540b2230 legacy=0x7f565406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56537fe700 1 -- 172.21.15.83:0/1537851950 >> v1:172.21.15.83:6789/0 conn(0x7f56540d2760 legacy=0x7f56540b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.516+0000 7f56537fe700 1 -- 172.21.15.83:0/1537851950 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540cd480 con 0x7f56540b5050 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56537fe700 1 --2- 172.21.15.83:0/1537851950 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f5654167560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f564c009dc0 tx=0x7f564c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453772b8e1bce06d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00e070 con 0x7f56540b5050 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c00bb20 con 0x7f56540b5050 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f5651ffb700 1 -- 172.21.15.83:0/1537851950 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00be90 con 0x7f56540b5050 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 -- 172.21.15.83:0/1537851950 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 msgr2=0x7f5654167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 --2- 172.21.15.83:0/1537851950 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f5654167560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f564c009dc0 tx=0x7f564c00b330 comp rx=0 tx=0).stop 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 -- 172.21.15.83:0/1537851950 shutdown_connections 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 --2- 172.21.15.83:0/1537851950 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f5654167560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 -- 172.21.15.83:0/1537851950 >> 172.21.15.83:0/1537851950 conn(0x7f56540d30a0 msgr2=0x7f56540d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 -- 172.21.15.83:0/1537851950 shutdown_connections 2023-11-28T11:12:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.517+0000 7f56593b0700 1 -- 172.21.15.83:0/1537851950 wait complete. 2023-11-28T11:12:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 Processor -- start 2023-11-28T11:12:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 -- start start 2023-11-28T11:12:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56540b2230 0x7f565414bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f56541f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56540d2760 0x7f56541f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56540cc8e0 con 0x7f56540d2760 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.519+0000 7f56593b0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56540ccbc0 con 0x7f56540b2230 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56593b0700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56540ccea0 con 0x7f56540b5050 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56540d2760 0x7f56541f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56540d2760 0x7f56541f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53872/0 (socket says 172.21.15.83:53872) 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 -- 172.21.15.83:0/4082783380 learned_addr learned my addr 172.21.15.83:0/4082783380 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 -- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 msgr2=0x7f56541f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56527fc700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f56541f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f5652ffd700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56540b2230 0x7f565414bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f56540b5050 0x7f56541f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 -- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56540b2230 msgr2=0x7f565414bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56540b2230 0x7f565414bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 -- 172.21.15.83:0/4082783380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c009a70 con 0x7f56540d2760 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56537fe700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56540d2760 0x7f56541f5810 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f564c009fd0 tx=0x7f564c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c003c20 con 0x7f56540d2760 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c01f070 con 0x7f56540d2760 2023-11-28T11:12:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56540cf360 con 0x7f56540d2760 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0098f0 con 0x7f56540d2760 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.520+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56540cf080 con 0x7f56540d2760 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.521+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f56540cf640 con 0x7f56540d2760 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f564c004030 con 0x7f56540d2760 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f563b7fe700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f56400682f0 0x7f564006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f564c087b80 con 0x7f56540d2760 2023-11-28T11:12:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f5652ffd700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f56400682f0 0x7f564006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f563b7fe700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f564006eb90 0x7f5640070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 --> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56400716b0 con 0x7f564006eb90 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f564c087fc0 con 0x7f56540d2760 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f56527fc700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f564006eb90 0x7f5640070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.522+0000 7f5652ffd700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f56400682f0 0x7f564006a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f56480015b0 tx=0x7f5648009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.523+0000 7f56527fc700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f564006eb90 0x7f5640070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.524+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== osd.6 v2:172.21.15.90:6800/103764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f56400716b0 con 0x7f564006eb90 2023-11-28T11:12:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 --> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f565415df10 con 0x7f564006eb90 2023-11-28T11:12:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f563b7fe700 1 -- 172.21.15.83:0/4082783380 <== osd.6 v2:172.21.15.90:6800/103764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f565415df10 con 0x7f564006eb90 2023-11-28T11:12:47.539 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:47.539 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:47.539 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f564006eb90 msgr2=0x7f5640070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.90:6800/103764,v1:172.21.15.90:6801/103764] conn(0x7f564006eb90 0x7f5640070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f56400682f0 msgr2=0x7f564006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f56400682f0 0x7f564006a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f56480015b0 tx=0x7f5648009040 comp rx=0 tx=0).stop 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56540d2760 msgr2=0x7f56541f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56593b0700 1 --2- 172.21.15.83:0/4082783380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56540d2760 0x7f56541f5810 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f564c009fd0 tx=0x7f564c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.538+0000 7f56537fe700 1 -- 172.21.15.83:0/4082783380 reap_dead start 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.539+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 shutdown_connections 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.539+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 >> 172.21.15.83:0/4082783380 conn(0x7f56540d30a0 msgr2=0x7f565415ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.539+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 shutdown_connections 2023-11-28T11:12:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.539+0000 7f56593b0700 1 -- 172.21.15.83:0/4082783380 wait complete. 2023-11-28T11:12:47.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T11:12:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.671+0000 7ff3300fd700 1 Processor -- start 2023-11-28T11:12:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.671+0000 7ff3300fd700 1 -- start start 2023-11-28T11:12:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff3300fd700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff32815c6d0 0x7ff3280ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff3300fd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 0x7ff3281602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff3300fd700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3280cbb30 con 0x7ff328160800 2023-11-28T11:12:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff3300fd700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3280701b0 con 0x7ff32815c6d0 2023-11-28T11:12:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff3300fd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280cf2f0 con 0x7ff3280cee70 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff32e69a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff328160800 0x7ff3280b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51738/0 (socket says 172.21.15.83:51738) 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff32e69a700 1 -- 172.21.15.83:0/1308841380 learned_addr learned my addr 172.21.15.83:0/1308841380 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff32d698700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 0x7ff3281602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.672+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212422210 0 0) 0x7ff3280cbb30 con 0x7ff328160800 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3140035b0 con 0x7ff328160800 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32de99700 1 --2- 172.21.15.83:0/1308841380 >> v2:172.21.15.90:3300/0 conn(0x7ff32815c6d0 0x7ff3280ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939220706 0 0) 0x7ff3140035b0 con 0x7ff328160800 2023-11-28T11:12:47.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 msgr2=0x7ff3281602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 0x7ff3281602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 >> v2:172.21.15.90:3300/0 conn(0x7ff32815c6d0 msgr2=0x7ff3280ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 --2- 172.21.15.83:0/1308841380 >> v2:172.21.15.90:3300/0 conn(0x7ff32815c6d0 0x7ff3280ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3281611f0 con 0x7ff328160800 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7ff3200019f0 con 0x7ff328160800 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 >> v1:172.21.15.83:6789/0 conn(0x7ff328160800 legacy=0x7ff3280b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff314003a00 0x7ff314003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff314004430 0x7ff3140048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 0x7ff314005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280701b0 con 0x7ff314004430 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280cf2f0 con 0x7ff314003a00 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31400ba40 con 0x7ff3280cee70 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff314004430 0x7ff3140048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32de99700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff314003a00 0x7ff314003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32e69a700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 0x7ff314005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 -- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 msgr2=0x7ff314005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3280cee70 0x7ff314005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 -- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff314003a00 msgr2=0x7ff314003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff314003a00 0x7ff314003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 -- 172.21.15.83:0/1308841380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140035b0 con 0x7ff314004430 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32d698700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff314004430 0x7ff3140048f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff324009b60 tx=0x7ff314005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf4875735f457e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.673+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400e660 con 0x7ff314004430 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32400e7c0 con 0x7ff314004430 2023-11-28T11:12:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff32ce97700 1 -- 172.21.15.83:0/1308841380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400eb30 con 0x7ff314004430 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff3300fd700 1 -- 172.21.15.83:0/1308841380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff314004430 msgr2=0x7ff3140048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff3300fd700 1 --2- 172.21.15.83:0/1308841380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff314004430 0x7ff3140048f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff324009b60 tx=0x7ff314005880 comp rx=0 tx=0).stop 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff32e69a700 1 -- 172.21.15.83:0/1308841380 reap_dead start 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff3300fd700 1 -- 172.21.15.83:0/1308841380 shutdown_connections 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff3300fd700 1 -- 172.21.15.83:0/1308841380 >> 172.21.15.83:0/1308841380 conn(0x7ff32806de20 msgr2=0x7ff32806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff3300fd700 1 -- 172.21.15.83:0/1308841380 shutdown_connections 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.674+0000 7ff3300fd700 1 -- 172.21.15.83:0/1308841380 wait complete. 2023-11-28T11:12:47.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.676+0000 7ff3300fd700 1 Processor -- start 2023-11-28T11:12:47.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.676+0000 7ff3300fd700 1 -- start start 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff314004430 0x7ff32815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 0x7ff328156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32815f160 0x7ff3280c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32815fdc0 con 0x7ff3281f2dc0 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32815fae0 con 0x7ff32815f160 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280c7f40 con 0x7ff314004430 2023-11-28T11:12:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 0x7ff328156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 0x7ff328156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53892/0 (socket says 172.21.15.83:53892) 2023-11-28T11:12:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 -- 172.21.15.83:0/660044360 learned_addr learned my addr 172.21.15.83:0/660044360 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32de99700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff314004430 0x7ff32815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32e69a700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32815f160 0x7ff3280c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 -- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff314004430 msgr2=0x7ff32815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff314004430 0x7ff32815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 -- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32815f160 msgr2=0x7ff3280c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32815f160 0x7ff3280c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 -- 172.21.15.83:0/660044360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328160f10 con 0x7ff3281f2dc0 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32de99700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff314004430 0x7ff32815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32e69a700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32815f160 0x7ff3280c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff32d698700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 0x7ff328156250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff3140057d0 tx=0x7ff3240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324021070 con 0x7ff3281f2dc0 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3240045d0 con 0x7ff3281f2dc0 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff328160c80 con 0x7ff3281f2dc0 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400e660 con 0x7ff3281f2dc0 2023-11-28T11:12:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.677+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3280cb4e0 con 0x7ff3281f2dc0 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.678+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32815fdc0 con 0x7ff3281f2dc0 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.679+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff32402e030 con 0x7ff3281f2dc0 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.679+0000 7ff31e7fc700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff30c0682f0 0x7ff30c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.679+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff324086d70 con 0x7ff3281f2dc0 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.679+0000 7ff32de99700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff30c0682f0 0x7ff30c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.680+0000 7ff32de99700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff30c0682f0 0x7ff30c06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff3280cb380 tx=0x7ff318008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.683+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3240537d0 con 0x7ff3281f2dc0 2023-11-28T11:12:47.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.867+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3280742b0 con 0x7ff30c0682f0 2023-11-28T11:12:47.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.867+0000 7ff31e7fc700 1 -- 172.21.15.83:0/660044360 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff3280742b0 con 0x7ff30c0682f0 2023-11-28T11:12:47.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff30c0682f0 msgr2=0x7ff30c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff30c0682f0 0x7ff30c06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff3280cb380 tx=0x7ff318008040 comp rx=0 tx=0).stop 2023-11-28T11:12:47.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 msgr2=0x7ff328156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 0x7ff328156250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff3140057d0 tx=0x7ff3240047c0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 shutdown_connections 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32815f160 0x7ff3280c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff314004430 0x7ff32815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff30c0682f0 0x7ff30c06a7a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 --2- 172.21.15.83:0/660044360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3281f2dc0 0x7ff328156250 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 >> 172.21.15.83:0/660044360 conn(0x7ff32806de20 msgr2=0x7ff3280b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 shutdown_connections 2023-11-28T11:12:47.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:47.869+0000 7ff3300fd700 1 -- 172.21.15.83:0/660044360 wait complete. 2023-11-28T11:12:47.872 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T11:12:47.882 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T11:12:47.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:48.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.005+0000 7f2b2fa4d700 1 Processor -- start 2023-11-28T11:12:48.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.005+0000 7f2b2fa4d700 1 -- start start 2023-11-28T11:12:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2fa4d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b2815cd30 0x7f2b280cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2fa4d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2fa4d700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2815d100 con 0x7f2b280b17e0 2023-11-28T11:12:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2fa4d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b280b2330 con 0x7f2b2815cd30 2023-11-28T11:12:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2fa4d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2806beb0 con 0x7f2b2806cf30 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b2815cd30 0x7f2b280cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b2815cd30 0x7f2b280cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53898/0 (socket says 172.21.15.83:53898) 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 -- 172.21.15.83:0/615615999 learned_addr learned my addr 172.21.15.83:0/615615999 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2cfe8700 1 --2- 172.21.15.83:0/615615999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 -- 172.21.15.83:0/615615999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 msgr2=0x7f2b280cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 --2- 172.21.15.83:0/615615999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 -- 172.21.15.83:0/615615999 >> v1:172.21.15.90:6789/0 conn(0x7f2b280b17e0 legacy=0x7f2b280b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 -- 172.21.15.83:0/615615999 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2806c170 con 0x7f2b2815cd30 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.006+0000 7f2b2d7e9700 1 --2- 172.21.15.83:0/615615999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b280cae10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2b180098f0 tx=0x7f2b18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7830e287c56925 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b1ffff700 1 -- 172.21.15.83:0/615615999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800b8a0 con 0x7f2b2815cd30 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b1ffff700 1 -- 172.21.15.83:0/615615999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800ba00 con 0x7f2b2815cd30 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b1ffff700 1 -- 172.21.15.83:0/615615999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800bd90 con 0x7f2b2815cd30 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/615615999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 msgr2=0x7f2b280cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 --2- 172.21.15.83:0/615615999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b280cae10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2b180098f0 tx=0x7f2b18009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:48.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/615615999 shutdown_connections 2023-11-28T11:12:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 --2- 172.21.15.83:0/615615999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b280cae10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 --2- 172.21.15.83:0/615615999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/615615999 >> 172.21.15.83:0/615615999 conn(0x7f2b280cc690 msgr2=0x7f2b280cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/615615999 shutdown_connections 2023-11-28T11:12:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.007+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/615615999 wait complete. 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.008+0000 7f2b2fa4d700 1 Processor -- start 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 -- start start 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b280b17e0 0x7f2b280ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b2815e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280c7270 con 0x7f2b2815cd30 2023-11-28T11:12:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280c7550 con 0x7f2b280b17e0 2023-11-28T11:12:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2fa4d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280c7830 con 0x7f2b2806cf30 2023-11-28T11:12:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2dfea700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b2815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2dfea700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b2815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53906/0 (socket says 172.21.15.83:53906) 2023-11-28T11:12:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.009+0000 7f2b2dfea700 1 -- 172.21.15.83:0/1491078953 learned_addr learned my addr 172.21.15.83:0/1491078953 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2cfe8700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b280b17e0 0x7f2b280ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2d7e9700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2dfea700 1 -- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 msgr2=0x7f2b280ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2dfea700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2b2806cf30 0x7f2b280ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2dfea700 1 -- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b280b17e0 msgr2=0x7f2b280ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2dfea700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b280b17e0 0x7f2b280ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2dfea700 1 -- 172.21.15.83:0/1491078953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18009550 con 0x7f2b2815cd30 2023-11-28T11:12:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2dfea700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b2815e850 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000cc50 tx=0x7f2b2000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2000ad80 con 0x7f2b2815cd30 2023-11-28T11:12:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2000ce50 con 0x7f2b2815cd30 2023-11-28T11:12:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2806c730 con 0x7f2b2815cd30 2023-11-28T11:12:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20003c40 con 0x7f2b2815cd30 2023-11-28T11:12:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.010+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2806c450 con 0x7f2b2815cd30 2023-11-28T11:12:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.011+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b2806ca10 con 0x7f2b2815cd30 2023-11-28T11:12:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.011+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b20010050 con 0x7f2b2815cd30 2023-11-28T11:12:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b1dffb700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2b0c0682f0 0x7f2b0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2b200858f0 con 0x7f2b2815cd30 2023-11-28T11:12:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b2d7e9700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2b0c0682f0 0x7f2b0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b1dffb700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7f2b0c06eb90 0x7f2b0c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 --> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c0716b0 con 0x7f2b0c06eb90 2023-11-28T11:12:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b2008a2a0 con 0x7f2b2815cd30 2023-11-28T11:12:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b2d7e9700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2b0c0682f0 0x7f2b0c06a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b18014040 tx=0x7f2b18008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.012+0000 7f2b2cfe8700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7f2b0c06eb90 0x7f2b0c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.013+0000 7f2b2cfe8700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7f2b0c06eb90 0x7f2b0c070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.013+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== osd.7 v2:172.21.15.90:6808/103765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2b0c0716b0 con 0x7f2b0c06eb90 2023-11-28T11:12:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 --> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2b2815d100 con 0x7f2b0c06eb90 2023-11-28T11:12:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b1dffb700 1 -- 172.21.15.83:0/1491078953 <== osd.7 v2:172.21.15.90:6808/103765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2b2815d100 con 0x7f2b0c06eb90 2023-11-28T11:12:48.029 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-28T11:12:48.029 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:48.029 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-28T11:12:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7f2b0c06eb90 msgr2=0x7f2b0c070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.90:6808/103765,v1:172.21.15.90:6809/103765] conn(0x7f2b0c06eb90 0x7f2b0c070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2b0c0682f0 msgr2=0x7f2b0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f2b0c0682f0 0x7f2b0c06a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b18014040 tx=0x7f2b18008030 comp rx=0 tx=0).stop 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 msgr2=0x7f2b2815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 --2- 172.21.15.83:0/1491078953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2815cd30 0x7f2b2815e850 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000cc50 tx=0x7f2b2000a340 comp rx=0 tx=0).stop 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2dfea700 1 -- 172.21.15.83:0/1491078953 reap_dead start 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 shutdown_connections 2023-11-28T11:12:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.028+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 >> 172.21.15.83:0/1491078953 conn(0x7f2b280cc690 msgr2=0x7f2b280b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.029+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 shutdown_connections 2023-11-28T11:12:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.029+0000 7f2b2fa4d700 1 -- 172.21.15.83:0/1491078953 wait complete. 2023-11-28T11:12:48.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T11:12:48.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.162+0000 7ff06c266700 1 Processor -- start 2023-11-28T11:12:48.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.162+0000 7ff06c266700 1 -- start start 2023-11-28T11:12:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff06c266700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff06406e630 0x7ff0640c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff06c266700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff0640c3120 0x7ff0640cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff06c266700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640c3540 con 0x7ff064160d70 2023-11-28T11:12:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff06c266700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640cc490 con 0x7ff06406e630 2023-11-28T11:12:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff06c266700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640cc770 con 0x7ff0640c3120 2023-11-28T11:12:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff069801700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff06406e630 0x7ff0640c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff069801700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff06406e630 0x7ff0640c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53914/0 (socket says 172.21.15.83:53914) 2023-11-28T11:12:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff069801700 1 -- 172.21.15.83:0/2814887880 learned_addr learned my addr 172.21.15.83:0/2814887880 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.163+0000 7ff06a803700 1 --2- 172.21.15.83:0/2814887880 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff0640c3120 0x7ff0640cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069801700 1 -- 172.21.15.83:0/2814887880 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff0640c3120 msgr2=0x7ff0640cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069801700 1 --2- 172.21.15.83:0/2814887880 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff0640c3120 0x7ff0640cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069801700 1 -- 172.21.15.83:0/2814887880 >> v1:172.21.15.90:6789/0 conn(0x7ff064160d70 legacy=0x7ff06406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069000700 1 -- 172.21.15.83:0/2814887880 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165193740 0 0) 0x7ff0640c3540 con 0x7ff064160d70 2023-11-28T11:12:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069801700 1 -- 172.21.15.83:0/2814887880 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640cca50 con 0x7ff06406e630 2023-11-28T11:12:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069801700 1 --2- 172.21.15.83:0/2814887880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff06406e630 0x7ff0640c2be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff0540098f0 tx=0x7ff054009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881e677ee7c23ee1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069000700 1 -- 172.21.15.83:0/2814887880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054011830 con 0x7ff06406e630 2023-11-28T11:12:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069000700 1 -- 172.21.15.83:0/2814887880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054011990 con 0x7ff06406e630 2023-11-28T11:12:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.164+0000 7ff069000700 1 -- 172.21.15.83:0/2814887880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054011d20 con 0x7ff06406e630 2023-11-28T11:12:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 -- 172.21.15.83:0/2814887880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff06406e630 msgr2=0x7ff0640c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 --2- 172.21.15.83:0/2814887880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff06406e630 0x7ff0640c2be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff0540098f0 tx=0x7ff054009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 -- 172.21.15.83:0/2814887880 shutdown_connections 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 --2- 172.21.15.83:0/2814887880 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff0640c3120 0x7ff0640cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 --2- 172.21.15.83:0/2814887880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff06406e630 0x7ff0640c2be0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 -- 172.21.15.83:0/2814887880 >> 172.21.15.83:0/2814887880 conn(0x7ff0640b2b60 msgr2=0x7ff0640b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 -- 172.21.15.83:0/2814887880 shutdown_connections 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.165+0000 7ff06c266700 1 -- 172.21.15.83:0/2814887880 wait complete. 2023-11-28T11:12:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.166+0000 7ff06c266700 1 Processor -- start 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.166+0000 7ff06c266700 1 -- start start 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff06406e630 0x7ff06415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 0x7ff0640b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff064160d70 0x7ff0640c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c90d0 con 0x7ff0640c3120 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c84e0 con 0x7ff06406e630 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c8730 con 0x7ff064160d70 2023-11-28T11:12:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 0x7ff0640b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 0x7ff0640b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53920/0 (socket says 172.21.15.83:53920) 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 -- 172.21.15.83:0/3147033035 learned_addr learned my addr 172.21.15.83:0/3147033035 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06a803700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff064160d70 0x7ff0640c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06a002700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff06406e630 0x7ff06415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 -- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff064160d70 msgr2=0x7ff0640c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff064160d70 0x7ff0640c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 -- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff06406e630 msgr2=0x7ff06415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff06406e630 0x7ff06415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 -- 172.21.15.83:0/3147033035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054009550 con 0x7ff0640c3120 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06a002700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff06406e630 0x7ff06415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff069801700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 0x7ff0640b1e00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff054009c40 tx=0x7ff054004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0540036a0 con 0x7ff0640c3120 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054003800 con 0x7ff0640c3120 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0640cd010 con 0x7ff0640c3120 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054003bb0 con 0x7ff0640c3120 2023-11-28T11:12:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.167+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0640ccd30 con 0x7ff0640c3120 2023-11-28T11:12:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.168+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0640cd2f0 con 0x7ff0640c3120 2023-11-28T11:12:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.169+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff05400f050 con 0x7ff0640c3120 2023-11-28T11:12:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.169+0000 7ff05a7fc700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff0480682f0 0x7ff04806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.169+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff0540872d0 con 0x7ff0640c3120 2023-11-28T11:12:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.169+0000 7ff06a002700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff0480682f0 0x7ff04806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.170+0000 7ff06a002700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff0480682f0 0x7ff04806a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff0640ca1c0 tx=0x7ff060009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.174+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff054053cf0 con 0x7ff0640c3120 2023-11-28T11:12:48.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.355+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0640742b0 con 0x7ff0480682f0 2023-11-28T11:12:48.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.355+0000 7ff05a7fc700 1 -- 172.21.15.83:0/3147033035 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff0640742b0 con 0x7ff0480682f0 2023-11-28T11:12:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff0480682f0 msgr2=0x7ff04806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff0480682f0 0x7ff04806a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff0640ca1c0 tx=0x7ff060009040 comp rx=0 tx=0).stop 2023-11-28T11:12:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 msgr2=0x7ff0640b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 0x7ff0640b1e00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff054009c40 tx=0x7ff054004750 comp rx=0 tx=0).stop 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 shutdown_connections 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff064160d70 0x7ff0640c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff06406e630 0x7ff06415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7ff0480682f0 0x7ff04806a7a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 --2- 172.21.15.83:0/3147033035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640c3120 0x7ff0640b1e00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.357+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 >> 172.21.15.83:0/3147033035 conn(0x7ff0640b2b60 msgr2=0x7ff0640cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.358+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 shutdown_connections 2023-11-28T11:12:48.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.358+0000 7ff06c266700 1 -- 172.21.15.83:0/3147033035 wait complete. 2023-11-28T11:12:48.360 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T11:12:48.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.489+0000 7f82e2701700 1 Processor -- start 2023-11-28T11:12:48.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82e2701700 1 -- start start 2023-11-28T11:12:48.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82e2701700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82dc15a8e0 0x7f82dc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82e2701700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc0b2ea0 0x7f82dc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82e2701700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc15f1d0 con 0x7f82dc15a510 2023-11-28T11:12:48.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82e2701700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc0cd780 con 0x7f82dc15a8e0 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82e2701700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b3320 con 0x7f82dc0b2ea0 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82db7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82dc15a8e0 0x7f82dc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82dc15a8e0 0x7f82dc0b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53926/0 (socket says 172.21.15.83:53926) 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 -- 172.21.15.83:0/4169098887 learned_addr learned my addr 172.21.15.83:0/4169098887 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.490+0000 7f82dbfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f82dc15a510 0x7f82dc0cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51786/0 (socket says 172.21.15.83:51786) 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82e0c9e700 1 --2- 172.21.15.83:0/4169098887 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc0b2ea0 0x7f82dc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 -- 172.21.15.83:0/4169098887 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc0b2ea0 msgr2=0x7f82dc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 --2- 172.21.15.83:0/4169098887 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc0b2ea0 0x7f82dc0b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 -- 172.21.15.83:0/4169098887 >> v1:172.21.15.83:6789/0 conn(0x7f82dc15a510 legacy=0x7f82dc0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 -- 172.21.15.83:0/4169098887 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc15eef0 con 0x7f82dc15a8e0 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82daffd700 1 -- 172.21.15.83:0/4169098887 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010644392 0 0) 0x7f82dc15f1d0 con 0x7f82dc15a510 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82db7fe700 1 --2- 172.21.15.83:0/4169098887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc0b2960 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f82d00098f0 tx=0x7f82d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a22b2056f21fb34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82daffd700 1 -- 172.21.15.83:0/4169098887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000b8a0 con 0x7f82dc15a8e0 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82daffd700 1 -- 172.21.15.83:0/4169098887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d000ba00 con 0x7f82dc15a8e0 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.491+0000 7f82daffd700 1 -- 172.21.15.83:0/4169098887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000bd90 con 0x7f82dc15a8e0 2023-11-28T11:12:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 -- 172.21.15.83:0/4169098887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 msgr2=0x7f82dc0b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 --2- 172.21.15.83:0/4169098887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc0b2960 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f82d00098f0 tx=0x7f82d0009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 -- 172.21.15.83:0/4169098887 shutdown_connections 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 --2- 172.21.15.83:0/4169098887 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc0b2ea0 0x7f82dc0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 --2- 172.21.15.83:0/4169098887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc0b2960 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 -- 172.21.15.83:0/4169098887 >> 172.21.15.83:0/4169098887 conn(0x7f82dc0cd960 msgr2=0x7f82dc0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 -- 172.21.15.83:0/4169098887 shutdown_connections 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.492+0000 7f82e2701700 1 -- 172.21.15.83:0/4169098887 wait complete. 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.493+0000 7f82e2701700 1 Processor -- start 2023-11-28T11:12:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.493+0000 7f82e2701700 1 -- start start 2023-11-28T11:12:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc0b2ea0 0x7f82dc0c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc15a510 0x7f82dc157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c6950 con 0x7f82dc15a8e0 2023-11-28T11:12:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c5a60 con 0x7f82dc0b2ea0 2023-11-28T11:12:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c5d40 con 0x7f82dc15a510 2023-11-28T11:12:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc1584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53934/0 (socket says 172.21.15.83:53934) 2023-11-28T11:12:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 -- 172.21.15.83:0/1567119422 learned_addr learned my addr 172.21.15.83:0/1567119422 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:12:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82db7fe700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc15a510 0x7f82dc157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82dbfff700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc0b2ea0 0x7f82dc0c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 -- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc15a510 msgr2=0x7f82dc157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc15a510 0x7f82dc157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 -- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc0b2ea0 msgr2=0x7f82dc0c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc0b2ea0 0x7f82dc0c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 -- 172.21.15.83:0/1567119422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0009550 con 0x7f82dc15a8e0 2023-11-28T11:12:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82db7fe700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc15a510 0x7f82dc157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e0c9e700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc1584f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f82d400cd70 tx=0x7f82d400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4003a20 con 0x7f82dc15a8e0 2023-11-28T11:12:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d4003b80 con 0x7f82dc15a8e0 2023-11-28T11:12:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc0cf0b0 con 0x7f82dc15a8e0 2023-11-28T11:12:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4004020 con 0x7f82dc15a8e0 2023-11-28T11:12:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.494+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc0cedd0 con 0x7f82dc15a8e0 2023-11-28T11:12:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.495+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc0cf390 con 0x7f82dc15a8e0 2023-11-28T11:12:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.496+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f82d401c020 con 0x7f82dc15a8e0 2023-11-28T11:12:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.496+0000 7f82d8ff9700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82c80682f0 0x7f82c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.496+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f82d4012070 con 0x7f82dc15a8e0 2023-11-28T11:12:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.496+0000 7f82dbfff700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82c80682f0 0x7f82c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.497+0000 7f82dbfff700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82c80682f0 0x7f82c806a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000f60 tx=0x7f82cc007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.500+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f82d4052d20 con 0x7f82dc15a8e0 2023-11-28T11:12:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.681+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82dc074320 con 0x7f82c80682f0 2023-11-28T11:12:48.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.683+0000 7f82d8ff9700 1 -- 172.21.15.83:0/1567119422 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73292 (secure 0 0 0) 0x7f82dc074320 con 0x7f82c80682f0 2023-11-28T11:12:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:12:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82c80682f0 msgr2=0x7f82c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82c80682f0 0x7f82c806a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000f60 tx=0x7f82cc007be0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 msgr2=0x7f82dc1584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc1584f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f82d400cd70 tx=0x7f82d400a570 comp rx=0 tx=0).stop 2023-11-28T11:12:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 shutdown_connections 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f82c80682f0 0x7f82c806a7a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82dc15a8e0 0x7f82dc1584f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc0b2ea0 0x7f82dc0c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 --2- 172.21.15.83:0/1567119422 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f82dc15a510 0x7f82dc157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.685+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 >> 172.21.15.83:0/1567119422 conn(0x7f82dc0cd960 msgr2=0x7f82dc06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.686+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 shutdown_connections 2023-11-28T11:12:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:12:48.686+0000 7f82e2701700 1 -- 172.21.15.83:0/1567119422 wait complete. 2023-11-28T11:12:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:12:48.698 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2023-11-28T11:12:47.625609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81568,"kb_used_data":7648,"kb_used_omap":0,"kb_used_meta":73856,"kb_avail":754893152,"statfs":{"total":773094113280,"available":773010587648,"internally_reserved":0,"allocated":7831552,"data_stored":4298950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75628544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.577535"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:40.387432+0000","last_change":"2023-11-28T11:08:30.845986+0000","last_active":"2023-11-28T11:12:40.387432+0000","last_peered":"2023-11-28T11:12:40.387432+0000","last_clean":"2023-11-28T11:12:40.387432+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:12:40.387432+0000","last_undegraded":"2023-11-28T11:12:40.387432+0000","last_fullsized":"2023-11-28T11:12:40.387432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:40.059749+0000","last_change":"2023-11-28T11:08:30.845429+0000","last_active":"2023-11-28T11:12:40.059749+0000","last_peered":"2023-11-28T11:12:40.059749+0000","last_clean":"2023-11-28T11:12:40.059749+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:12:40.059749+0000","last_undegraded":"2023-11-28T11:12:40.059749+0000","last_fullsized":"2023-11-28T11:12:40.059749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:17.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:40.386235+0000","last_change":"2023-11-28T11:08:30.847461+0000","last_active":"2023-11-28T11:12:40.386235+0000","last_peered":"2023-11-28T11:12:40.386235+0000","last_clean":"2023-11-28T11:12:40.386235+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:12:40.386235+0000","last_undegraded":"2023-11-28T11:12:40.386235+0000","last_fullsized":"2023-11-28T11:12:40.386235+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":202,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:36.898761+0000","last_change":"2023-11-28T11:08:30.848993+0000","last_active":"2023-11-28T11:12:36.898761+0000","last_peered":"2023-11-28T11:12:36.898761+0000","last_clean":"2023-11-28T11:12:36.898761+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:12:36.898761+0000","last_undegraded":"2023-11-28T11:12:36.898761+0000","last_fullsized":"2023-11-28T11:12:36.898761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:21.411104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:40.260547+0000","last_change":"2023-11-28T11:08:30.888547+0000","last_active":"2023-11-28T11:12:40.260547+0000","last_peered":"2023-11-28T11:12:40.260547+0000","last_clean":"2023-11-28T11:12:40.260547+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:12:40.260547+0000","last_undegraded":"2023-11-28T11:12:40.260547+0000","last_fullsized":"2023-11-28T11:12:40.260547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:33.355036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:36.898701+0000","last_change":"2023-11-28T11:08:30.888659+0000","last_active":"2023-11-28T11:12:36.898701+0000","last_peered":"2023-11-28T11:12:36.898701+0000","last_clean":"2023-11-28T11:12:36.898701+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:12:36.898701+0000","last_undegraded":"2023-11-28T11:12:36.898701+0000","last_fullsized":"2023-11-28T11:12:36.898701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:45.220060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:40.382465+0000","last_change":"2023-11-28T11:08:30.888702+0000","last_active":"2023-11-28T11:12:40.382465+0000","last_peered":"2023-11-28T11:12:40.382465+0000","last_clean":"2023-11-28T11:12:40.382465+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:12:40.382465+0000","last_undegraded":"2023-11-28T11:12:40.382465+0000","last_fullsized":"2023-11-28T11:12:40.382465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:07.679730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:36.899154+0000","last_change":"2023-11-28T11:08:30.888796+0000","last_active":"2023-11-28T11:12:36.899154+0000","last_peered":"2023-11-28T11:12:36.899154+0000","last_clean":"2023-11-28T11:12:36.899154+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:12:36.899154+0000","last_undegraded":"2023-11-28T11:12:36.899154+0000","last_fullsized":"2023-11-28T11:12:36.899154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:08.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:36.899798+0000","last_change":"2023-11-28T11:08:30.844596+0000","last_active":"2023-11-28T11:12:36.899798+0000","last_peered":"2023-11-28T11:12:36.899798+0000","last_clean":"2023-11-28T11:12:36.899798+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:12:36.899798+0000","last_undegraded":"2023-11-28T11:12:36.899798+0000","last_fullsized":"2023-11-28T11:12:36.899798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:29.830252+0000","last_clean_scrub_stamp":"2023-11-28T11:08:29.830252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:59.993431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11000,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360840,"statfs":{"total":96636764160,"available":96625500160,"internally_reserved":0,"allocated":1425408,"data_stored":980485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.746,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.46700000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.48599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.60499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9840,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362000,"statfs":{"total":96636764160,"available":96626688000,"internally_reserved":0,"allocated":827392,"data_stored":388756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70199999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":1.544,"15min":1.544},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66000000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.502},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":1.665,"15min":1.665},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.70599999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.443},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69599999999999995,"5min":1.762,"15min":1.762},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.39800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.002,"15min":1.002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10936,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360904,"statfs":{"total":96636764160,"available":96625565696,"internally_reserved":0,"allocated":1425408,"data_stored":980485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.247,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54900000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"min":{"1min":0.25800000000000001,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.251,"5min":0.187,"15min":0.187},"max":{"1min":0.54400000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39800000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63800000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.627,"5min":1.363,"15min":1.363},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90600000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":1.48,"15min":1.48},"last":0.58499999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.5},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.57499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10032,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361808,"statfs":{"total":96636764160,"available":96626491392,"internally_reserved":0,"allocated":827392,"data_stored":388756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.222},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.33300000000000002},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.248}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55600000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53000000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.312},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57499999999999996,"5min":0.871,"15min":0.871},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.379}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80100000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68600000000000005,"15min":0.754},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.877,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.434},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.45400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9908,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361932,"statfs":{"total":96636764160,"available":96626618368,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33300000000000002},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32700000000000001},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.628,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.40200000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.51000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71699999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79100000000000004,"5min":1.024,"15min":1.024},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80500000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.76600000000000001,"5min":1.05,"15min":1.05},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9972,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361868,"statfs":{"total":96636764160,"available":96626552832,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46800000000000003,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.249},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.48999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59399999999999997,"5min":1.054,"15min":1.054},"last":0.224},{"interface":"front","average":{"1min":0.378,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56299999999999994,"5min":1.147,"15min":1.147},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55700000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.49299999999999999,"15min":0.59099999999999997},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":5.133,"15min":5.133},"last":0.28399999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.67500000000000004,"5min":0.877,"15min":0.877},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.498}]},{"osd":6,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":1.194,"15min":1.194},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":1.077,"15min":1.077},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361740,"statfs":{"total":96636764160,"available":96626421760,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.78700000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.754}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.65800000000000003,"5min":0.753,"15min":0.753},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.54500000000000004,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72199999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.62},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.77000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67700000000000005,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78700000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.68700000000000006,"15min":0.84699999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72299999999999998,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.34999999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71399999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65400000000000003,"15min":0.77000000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":6.601,"15min":6.601},"last":0.442}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40899999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48299999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71799999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.53500000000000003},"min":{"1min":0.45400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:48.703 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.704 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.704 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.705 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.705 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.705 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.706 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.706 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.706 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T11:08:29.830252+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=29, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=43, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:48.707 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T11:13:08.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:13:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.839+0000 7fb970fd1700 1 Processor -- start 2023-11-28T11:13:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.839+0000 7fb970fd1700 1 -- start start 2023-11-28T11:13:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb970fd1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 0x7fb96c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb970fd1700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb96c0b6350 0x7fb96c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb970fd1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96c0cda10 con 0x7fb96c15c9e0 2023-11-28T11:13:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb970fd1700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c0cdcf0 con 0x7fb96c0b6350 2023-11-28T11:13:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb970fd1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0d0c60 con 0x7fb96c0b5ee0 2023-11-28T11:13:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb96a59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb96c15c9e0 0x7fb96c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51798/0 (socket says 172.21.15.83:51798) 2023-11-28T11:13:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.840+0000 7fb96a59c700 1 -- 172.21.15.83:0/2289830632 learned_addr learned my addr 172.21.15.83:0/2289830632 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:13:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156953693 0 0) 0x7fb96c0cda10 con 0x7fb96c15c9e0 2023-11-28T11:13:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96ad9d700 1 --2- 172.21.15.83:0/2289830632 >> v2:172.21.15.90:3300/0 conn(0x7fb96c0b6350 0x7fb96c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9540035b0 con 0x7fb96c15c9e0 2023-11-28T11:13:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb969d9b700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 0x7fb96c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1418861852 0 0) 0x7fb9540035b0 con 0x7fb96c15c9e0 2023-11-28T11:13:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 msgr2=0x7fb96c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 0x7fb96c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 >> v2:172.21.15.90:3300/0 conn(0x7fb96c0b6350 msgr2=0x7fb96c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 --2- 172.21.15.83:0/2289830632 >> v2:172.21.15.90:3300/0 conn(0x7fb96c0b6350 0x7fb96c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0d0f40 con 0x7fb96c15c9e0 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1696683940 0 0) 0x7fb95c001670 con 0x7fb96c15c9e0 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 >> v1:172.21.15.83:6789/0 conn(0x7fb96c15c9e0 legacy=0x7fb96c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb954003a00 0x7fb954003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb954004430 0x7fb9540048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 0x7fb954004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0cdcf0 con 0x7fb954004430 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0d0c60 con 0x7fb954003a00 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95400ba10 con 0x7fb96c0b5ee0 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb969d9b700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb954004430 0x7fb9540048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96a59c700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb954003a00 0x7fb954003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.841+0000 7fb96ad9d700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 0x7fb954004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb969d9b700 1 -- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 msgr2=0x7fb954004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb969d9b700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb96c0b5ee0 0x7fb954004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb969d9b700 1 -- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb954003a00 msgr2=0x7fb954003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb969d9b700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb954003a00 0x7fb954003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb969d9b700 1 -- 172.21.15.83:0/2289830632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540035b0 con 0x7fb954004430 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb96a59c700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb954003a00 0x7fb954003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb969d9b700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb954004430 0x7fb9540048f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb960009b60 tx=0x7fb954005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058291b16866e49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96000e6c0 con 0x7fb954004430 2023-11-28T11:13:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96000e820 con 0x7fb954004430 2023-11-28T11:13:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb96959a700 1 -- 172.21.15.83:0/2289830632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96000eb90 con 0x7fb954004430 2023-11-28T11:13:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb970fd1700 1 -- 172.21.15.83:0/2289830632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb954004430 msgr2=0x7fb9540048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb970fd1700 1 --2- 172.21.15.83:0/2289830632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb954004430 0x7fb9540048f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb960009b60 tx=0x7fb954005880 comp rx=0 tx=0).stop 2023-11-28T11:13:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb96ad9d700 1 -- 172.21.15.83:0/2289830632 reap_dead start 2023-11-28T11:13:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb970fd1700 1 -- 172.21.15.83:0/2289830632 shutdown_connections 2023-11-28T11:13:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.842+0000 7fb970fd1700 1 -- 172.21.15.83:0/2289830632 >> 172.21.15.83:0/2289830632 conn(0x7fb96c0cfb10 msgr2=0x7fb96c0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.843+0000 7fb970fd1700 1 -- 172.21.15.83:0/2289830632 shutdown_connections 2023-11-28T11:13:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.843+0000 7fb970fd1700 1 -- 172.21.15.83:0/2289830632 wait complete. 2023-11-28T11:13:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.844+0000 7fb970fd1700 1 Processor -- start 2023-11-28T11:13:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.844+0000 7fb970fd1700 1 -- start start 2023-11-28T11:13:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb970fd1700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb954004430 0x7fb96c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb970fd1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb96c152630 0x7fb96c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb970fd1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 0x7fb96c0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb970fd1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c154b40 con 0x7fb96c0b4210 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb970fd1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0ca6a0 con 0x7fb96c152630 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb970fd1700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0ca2e0 con 0x7fb954004430 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 0x7fb96c0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 0x7fb96c0c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53954/0 (socket says 172.21.15.83:53954) 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 -- 172.21.15.83:0/328045351 learned_addr learned my addr 172.21.15.83:0/328045351 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:13:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb969d9b700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb96c152630 0x7fb96c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96a59c700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb954004430 0x7fb96c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 -- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb954004430 msgr2=0x7fb96c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb954004430 0x7fb96c0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 -- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb96c152630 msgr2=0x7fb96c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb96c152630 0x7fb96c0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96ad9d700 1 -- 172.21.15.83:0/328045351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0cd6e0 con 0x7fb96c0b4210 2023-11-28T11:13:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb96a59c700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb954004430 0x7fb96c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.845+0000 7fb969d9b700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb96c152630 0x7fb96c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.846+0000 7fb96ad9d700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 0x7fb96c0c8b20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb96c066e10 tx=0x7fb964011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.846+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400e040 con 0x7fb96c0b4210 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.846+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96400f040 con 0x7fb96c0b4210 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.846+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c0d1220 con 0x7fb96c0b4210 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.846+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400fde0 con 0x7fb96c0b4210 2023-11-28T11:13:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.846+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c0cf670 con 0x7fb96c0b4210 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.847+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c0ca7e0 con 0x7fb96c0b4210 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.847+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb964022460 con 0x7fb96c0b4210 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.848+0000 7fb95affd700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb94c0682f0 0x7fb94c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.848+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb9640860b0 con 0x7fb96c0b4210 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.848+0000 7fb96a59c700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb94c0682f0 0x7fb94c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.848+0000 7fb96a59c700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb94c0682f0 0x7fb94c06a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000fb0 tx=0x7fb95c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:08.852+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb96c0ca7e0 con 0x7fb96c0b4210 2023-11-28T11:13:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.031+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 --> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb96c0ca7e0 con 0x7fb94c0682f0 2023-11-28T11:13:09.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.033+0000 7fb95affd700 1 -- 172.21.15.83:0/328045351 <== mgr.4108 v2:172.21.15.83:6832/103263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73389 (secure 0 0 0) 0x7fb96c0ca7e0 con 0x7fb94c0682f0 2023-11-28T11:13:09.036 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:13:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.035+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb94c0682f0 msgr2=0x7fb94c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.035+0000 7fb970fd1700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb94c0682f0 0x7fb94c06a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000fb0 tx=0x7fb95c009040 comp rx=0 tx=0).stop 2023-11-28T11:13:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.035+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 msgr2=0x7fb96c0c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.035+0000 7fb970fd1700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 0x7fb96c0c8b20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb96c066e10 tx=0x7fb964011f80 comp rx=0 tx=0).stop 2023-11-28T11:13:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 shutdown_connections 2023-11-28T11:13:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fb94c0682f0 0x7fb94c06a7a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0b4210 0x7fb96c0c8b20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb954004430 0x7fb96c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 --2- 172.21.15.83:0/328045351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb96c152630 0x7fb96c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 >> 172.21.15.83:0/328045351 conn(0x7fb96c0cfb10 msgr2=0x7fb96c0b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:09.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 shutdown_connections 2023-11-28T11:13:09.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.036+0000 7fb970fd1700 1 -- 172.21.15.83:0/328045351 wait complete. 2023-11-28T11:13:09.039 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-28T11:13:09.049 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2023-11-28T11:13:07.629700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81632,"kb_used_data":7648,"kb_used_omap":0,"kb_used_meta":73920,"kb_avail":754893088,"statfs":{"total":773094113280,"available":773010522112,"internally_reserved":0,"allocated":7831552,"data_stored":4298950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75694080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002599"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:55.183956+0000","last_change":"2023-11-28T11:12:55.183918+0000","last_active":"2023-11-28T11:12:55.183956+0000","last_peered":"2023-11-28T11:12:55.183956+0000","last_clean":"2023-11-28T11:12:55.183956+0000","last_became_active":"2023-11-28T11:08:30.845657+0000","last_became_peered":"2023-11-28T11:08:30.845657+0000","last_unstale":"2023-11-28T11:12:55.183956+0000","last_undegraded":"2023-11-28T11:12:55.183956+0000","last_fullsized":"2023-11-28T11:12:55.183956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:55.183801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:55.183801+0000","last_clean_scrub_stamp":"2023-11-28T11:12:55.183801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:36.271142+0000","scrub_duration":0.0038560360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:46.664865+0000","last_change":"2023-11-28T11:12:46.664845+0000","last_active":"2023-11-28T11:12:46.664865+0000","last_peered":"2023-11-28T11:12:46.664865+0000","last_clean":"2023-11-28T11:12:46.664865+0000","last_became_active":"2023-11-28T11:08:30.845052+0000","last_became_peered":"2023-11-28T11:08:30.845052+0000","last_unstale":"2023-11-28T11:12:46.664865+0000","last_undegraded":"2023-11-28T11:12:46.664865+0000","last_fullsized":"2023-11-28T11:12:46.664865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:46.664769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:46.664769+0000","last_clean_scrub_stamp":"2023-11-28T11:12:46.664769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:40.382258+0000","scrub_duration":0.0039163519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":68,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:47.496200+0000","last_change":"2023-11-28T11:12:47.496165+0000","last_active":"2023-11-28T11:12:47.496200+0000","last_peered":"2023-11-28T11:12:47.496200+0000","last_clean":"2023-11-28T11:12:47.496200+0000","last_became_active":"2023-11-28T11:08:30.847138+0000","last_became_peered":"2023-11-28T11:08:30.847138+0000","last_unstale":"2023-11-28T11:12:47.496200+0000","last_undegraded":"2023-11-28T11:12:47.496200+0000","last_fullsized":"2023-11-28T11:12:47.496200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:47.496041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:47.496041+0000","last_clean_scrub_stamp":"2023-11-28T11:12:47.496041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:19.780102+0000","scrub_duration":0.003714658,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'119","reported_seq":219,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:57.676814+0000","last_change":"2023-11-28T11:12:57.676761+0000","last_active":"2023-11-28T11:12:57.676814+0000","last_peered":"2023-11-28T11:12:57.676814+0000","last_clean":"2023-11-28T11:12:57.676814+0000","last_became_active":"2023-11-28T11:08:30.848120+0000","last_became_peered":"2023-11-28T11:08:30.848120+0000","last_unstale":"2023-11-28T11:12:57.676814+0000","last_undegraded":"2023-11-28T11:12:57.676814+0000","last_fullsized":"2023-11-28T11:12:57.676814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'119","last_scrub_stamp":"2023-11-28T11:12:57.676647+0000","last_deep_scrub":"68'119","last_deep_scrub_stamp":"2023-11-28T11:12:57.676647+0000","last_clean_scrub_stamp":"2023-11-28T11:12:57.676647+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:38.763987+0000","scrub_duration":0.007537668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":67,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:56.680736+0000","last_change":"2023-11-28T11:12:56.680694+0000","last_active":"2023-11-28T11:12:56.680736+0000","last_peered":"2023-11-28T11:12:56.680736+0000","last_clean":"2023-11-28T11:12:56.680736+0000","last_became_active":"2023-11-28T11:08:30.888211+0000","last_became_peered":"2023-11-28T11:08:30.888211+0000","last_unstale":"2023-11-28T11:12:56.680736+0000","last_undegraded":"2023-11-28T11:12:56.680736+0000","last_fullsized":"2023-11-28T11:12:56.680736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:56.680577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:56.680577+0000","last_clean_scrub_stamp":"2023-11-28T11:12:56.680577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:11.567354+0000","scrub_duration":0.0040576470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:55.662177+0000","last_change":"2023-11-28T11:12:55.662146+0000","last_active":"2023-11-28T11:12:55.662177+0000","last_peered":"2023-11-28T11:12:55.662177+0000","last_clean":"2023-11-28T11:12:55.662177+0000","last_became_active":"2023-11-28T11:08:30.888383+0000","last_became_peered":"2023-11-28T11:08:30.888383+0000","last_unstale":"2023-11-28T11:12:55.662177+0000","last_undegraded":"2023-11-28T11:12:55.662177+0000","last_fullsized":"2023-11-28T11:12:55.662177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:55.662042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:55.662042+0000","last_clean_scrub_stamp":"2023-11-28T11:12:55.662042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:06:11.467064+0000","scrub_duration":0.0041919640000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":73,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:49.249936+0000","last_change":"2023-11-28T11:12:49.249901+0000","last_active":"2023-11-28T11:12:49.249936+0000","last_peered":"2023-11-28T11:12:49.249936+0000","last_clean":"2023-11-28T11:12:49.249936+0000","last_became_active":"2023-11-28T11:08:30.888400+0000","last_became_peered":"2023-11-28T11:08:30.888400+0000","last_unstale":"2023-11-28T11:12:49.249936+0000","last_undegraded":"2023-11-28T11:12:49.249936+0000","last_fullsized":"2023-11-28T11:12:49.249936+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:49.249750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:49.249750+0000","last_clean_scrub_stamp":"2023-11-28T11:12:49.249750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:19.246306+0000","scrub_duration":0.0040186939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:50.063601+0000","last_change":"2023-11-28T11:12:50.063557+0000","last_active":"2023-11-28T11:12:50.063601+0000","last_peered":"2023-11-28T11:12:50.063601+0000","last_clean":"2023-11-28T11:12:50.063601+0000","last_became_active":"2023-11-28T11:08:30.888331+0000","last_became_peered":"2023-11-28T11:08:30.888331+0000","last_unstale":"2023-11-28T11:12:50.063601+0000","last_undegraded":"2023-11-28T11:12:50.063601+0000","last_fullsized":"2023-11-28T11:12:50.063601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:50.063455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:50.063455+0000","last_clean_scrub_stamp":"2023-11-28T11:12:50.063455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:36.946495+0000","scrub_duration":0.0041813600000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:52.108774+0000","last_change":"2023-11-28T11:12:52.108709+0000","last_active":"2023-11-28T11:12:52.108774+0000","last_peered":"2023-11-28T11:12:52.108774+0000","last_clean":"2023-11-28T11:12:52.108774+0000","last_became_active":"2023-11-28T11:08:30.844299+0000","last_became_peered":"2023-11-28T11:08:30.844299+0000","last_unstale":"2023-11-28T11:12:52.108774+0000","last_undegraded":"2023-11-28T11:12:52.108774+0000","last_fullsized":"2023-11-28T11:12:52.108774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:52.108598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:52.108598+0000","last_clean_scrub_stamp":"2023-11-28T11:12:52.108598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:22.105155+0000","scrub_duration":0.003901346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11064,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360776,"statfs":{"total":96636764160,"available":96625434624,"internally_reserved":0,"allocated":1425408,"data_stored":980485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.746,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.46700000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.502},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.48599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.372},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.58399999999999996},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9840,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362000,"statfs":{"total":96636764160,"available":96626688000,"internally_reserved":0,"allocated":827392,"data_stored":388756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70199999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":1.544,"15min":1.544},"last":0.55200000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":1.337,"15min":1.337},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66000000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":1.665,"15min":1.665},"last":0.58199999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.70599999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69599999999999995,"5min":1.762,"15min":1.762},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71399999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59599999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.39800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.002,"15min":1.002},"last":0.622},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.504},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10936,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360904,"statfs":{"total":96636764160,"available":96625565696,"internally_reserved":0,"allocated":1425408,"data_stored":980485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.247,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54900000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"min":{"1min":0.25800000000000001,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.52100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.251,"5min":0.187,"15min":0.187},"max":{"1min":0.54400000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39800000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63800000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.627,"5min":1.363,"15min":1.363},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.66200000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83199999999999996,"5min":1.54,"15min":1.54},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90600000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88500000000000001,"5min":1.48,"15min":1.48},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.623},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10032,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361808,"statfs":{"total":96636764160,"available":96626491392,"internally_reserved":0,"allocated":827392,"data_stored":388756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.33300000000000002},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55600000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53000000000000003,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57499999999999996,"5min":0.871,"15min":0.871},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65000000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80100000000000005,"5min":3.6429999999999998,"15min":3.6429999999999998},"last":0.621}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68600000000000005,"15min":0.754},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.877,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63500000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.56100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9908,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361932,"statfs":{"total":96636764160,"available":96626618368,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33300000000000002},"min":{"1min":0.252,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.27600000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32700000000000001},"min":{"1min":0.247,"5min":0.22,"15min":0.22},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.628,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.374},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.51000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71699999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79100000000000004,"5min":1.024,"15min":1.024},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80500000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.76600000000000001,"5min":1.05,"15min":1.05},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9972,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361868,"statfs":{"total":96636764160,"available":96626552832,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46800000000000003,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.249},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.48999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.33400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59399999999999997,"5min":1.054,"15min":1.054},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56299999999999994,"5min":1.147,"15min":1.147},"last":0.42999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55700000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.49299999999999999,"15min":0.59099999999999997},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":5.133,"15min":5.133},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.67500000000000004,"5min":0.877,"15min":0.877},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":1.194,"15min":1.194},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":1.077,"15min":1.077},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361740,"statfs":{"total":96636764160,"available":96626421760,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.69799999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.59199999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.749},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.65800000000000003,"5min":0.753,"15min":0.753},"last":0.498},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.54500000000000004,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72199999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":390117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67700000000000005,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78700000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.68700000000000006,"15min":0.84699999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72299999999999998,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71399999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65400000000000003,"15min":0.77000000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":6.601,"15min":6.601},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40899999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48299999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71799999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.41899999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.439},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.53500000000000003},"min":{"1min":0.45400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53600000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:13:09.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T11:13:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.170+0000 7fdc2ad0d700 1 Processor -- start 2023-11-28T11:13:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.170+0000 7fdc2ad0d700 1 -- start start 2023-11-28T11:13:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc2ad0d700 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdc240aee20 0x7fdc240b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc2ad0d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc2406c370 0x7fdc2406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc2ad0d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240b5b00 con 0x7fdc240af1f0 2023-11-28T11:13:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc2ad0d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2406c800 con 0x7fdc2406c370 2023-11-28T11:13:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc2ad0d700 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2406cae0 con 0x7fdc240aee20 2023-11-28T11:13:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc23fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc2406c370 0x7fdc2406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc292aa700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdc240af1f0 0x7fdc240cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51822/0 (socket says 172.21.15.83:51822) 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc292aa700 1 -- 172.21.15.83:0/2064949929 learned_addr learned my addr 172.21.15.83:0/2064949929 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc28aa9700 1 --2- 172.21.15.83:0/2064949929 >> v2:172.21.15.90:3300/0 conn(0x7fdc240aee20 0x7fdc240b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc23fff700 1 -- 172.21.15.83:0/2064949929 >> v2:172.21.15.90:3300/0 conn(0x7fdc240aee20 msgr2=0x7fdc240b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc23fff700 1 --2- 172.21.15.83:0/2064949929 >> v2:172.21.15.90:3300/0 conn(0x7fdc240aee20 0x7fdc240b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc23fff700 1 -- 172.21.15.83:0/2064949929 >> v1:172.21.15.83:6789/0 conn(0x7fdc240af1f0 legacy=0x7fdc240cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc23fff700 1 -- 172.21.15.83:0/2064949929 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2406cdc0 con 0x7fdc2406c370 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc237fe700 1 -- 172.21.15.83:0/2064949929 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31124062 0 0) 0x7fdc1c001490 con 0x7fdc240af1f0 2023-11-28T11:13:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc28aa9700 1 --2- 172.21.15.83:0/2064949929 >> v2:172.21.15.90:3300/0 conn(0x7fdc240aee20 0x7fdc240b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc23fff700 1 --2- 172.21.15.83:0/2064949929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2406ed80 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdc240715b0 tx=0x7fdc10009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f389bf06e29598f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc237fe700 1 -- 172.21.15.83:0/2064949929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10011830 con 0x7fdc2406c370 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.171+0000 7fdc237fe700 1 -- 172.21.15.83:0/2064949929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10011990 con 0x7fdc2406c370 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc237fe700 1 -- 172.21.15.83:0/2064949929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10011d20 con 0x7fdc2406c370 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2064949929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 msgr2=0x7fdc2406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2064949929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2406ed80 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdc240715b0 tx=0x7fdc10009b20 comp rx=0 tx=0).stop 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2064949929 shutdown_connections 2023-11-28T11:13:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2064949929 >> v2:172.21.15.90:3300/0 conn(0x7fdc240aee20 0x7fdc240b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2064949929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2406ed80 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2064949929 >> 172.21.15.83:0/2064949929 conn(0x7fdc240b2b60 msgr2=0x7fdc240b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2064949929 shutdown_connections 2023-11-28T11:13:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.172+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2064949929 wait complete. 2023-11-28T11:13:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.173+0000 7fdc2ad0d700 1 Processor -- start 2023-11-28T11:13:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.173+0000 7fdc2ad0d700 1 -- start start 2023-11-28T11:13:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc2ad0d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc2ad0d700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc240aee20 0x7fdc24152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc2ad0d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc240af1f0 0x7fdc24166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc2ad0d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc24152a50 con 0x7fdc2406c370 2023-11-28T11:13:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc2ad0d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2414f490 con 0x7fdc240af1f0 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc2ad0d700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc24150a00 con 0x7fdc240aee20 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2414e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53964/0 (socket says 172.21.15.83:53964) 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 -- 172.21.15.83:0/2425408262 learned_addr learned my addr 172.21.15.83:0/2425408262 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 -- 172.21.15.83:0/2425408262 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc240aee20 msgr2=0x7fdc24152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc240aee20 0x7fdc24152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 -- 172.21.15.83:0/2425408262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc240af1f0 msgr2=0x7fdc24166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.174+0000 7fdc28aa9700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc240af1f0 0x7fdc24166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc28aa9700 1 -- 172.21.15.83:0/2425408262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100094a0 con 0x7fdc2406c370 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc28aa9700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2414e5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c820 tx=0x7fdc1800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18005c20 con 0x7fdc2406c370 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18005d80 con 0x7fdc2406c370 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2406fc70 con 0x7fdc2406c370 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18011820 con 0x7fdc2406c370 2023-11-28T11:13:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.175+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2406d0a0 con 0x7fdc2406c370 2023-11-28T11:13:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.176+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2406d380 con 0x7fdc2406c370 2023-11-28T11:13:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.176+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc18020070 con 0x7fdc2406c370 2023-11-28T11:13:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.177+0000 7fdc217fa700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdc080682f0 0x7fdc0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.177+0000 7fdc23fff700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdc080682f0 0x7fdc0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.177+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fdc18085ca0 con 0x7fdc2406c370 2023-11-28T11:13:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.177+0000 7fdc23fff700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdc080682f0 0x7fdc0806a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc24070d10 tx=0x7fdc10010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.181+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdc180526c0 con 0x7fdc2406c370 2023-11-28T11:13:09.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.361+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdc240742b0 con 0x7fdc2406c370 2023-11-28T11:13:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.364+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdc180526c0 con 0x7fdc2406c370 2023-11-28T11:13:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.365+0000 7fdc217fa700 1 -- 172.21.15.83:0/2425408262 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdc18055ce0 con 0x7fdc2406c370 2023-11-28T11:13:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdc080682f0 msgr2=0x7fdc0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdc080682f0 0x7fdc0806a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc24070d10 tx=0x7fdc10010000 comp rx=0 tx=0).stop 2023-11-28T11:13:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 msgr2=0x7fdc2414e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2414e5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c820 tx=0x7fdc1800cbe0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 shutdown_connections 2023-11-28T11:13:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc240af1f0 0x7fdc24166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7fdc080682f0 0x7fdc0806a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2406c370 0x7fdc2414e5e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 --2- 172.21.15.83:0/2425408262 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc240aee20 0x7fdc24152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 >> 172.21.15.83:0/2425408262 conn(0x7fdc240b2b60 msgr2=0x7fdc240ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 shutdown_connections 2023-11-28T11:13:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.368+0000 7fdc2ad0d700 1 -- 172.21.15.83:0/2425408262 wait complete. 2023-11-28T11:13:09.382 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:13:09.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:13:09.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.500+0000 7f89b3fa2700 1 Processor -- start 2023-11-28T11:13:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.500+0000 7f89b3fa2700 1 -- start start 2023-11-28T11:13:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b3fa2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89ac15c820 0x7f89ac0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b3fa2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac06ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b3fa2700 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ac0ca430 con 0x7f89ac06b290 2023-11-28T11:13:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b3fa2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac06a930 con 0x7f89ac15c820 2023-11-28T11:13:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b3fa2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0ca710 con 0x7f89ac160fe0 2023-11-28T11:13:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89ac15c820 0x7f89ac0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89ac15c820 0x7f89ac0c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53972/0 (socket says 172.21.15.83:53972) 2023-11-28T11:13:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 -- 172.21.15.83:0/26281776 learned_addr learned my addr 172.21.15.83:0/26281776 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:13:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b153d700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac06ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 -- 172.21.15.83:0/26281776 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 msgr2=0x7f89ac06ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac06ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 -- 172.21.15.83:0/26281776 >> v1:172.21.15.90:6789/0 conn(0x7f89ac06b290 legacy=0x7f89ac06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b1d3e700 1 -- 172.21.15.83:0/26281776 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac06a270 con 0x7f89ac15c820 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.501+0000 7f89b153d700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac06ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b1d3e700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac0c9b30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f89a80098f0 tx=0x7f89a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559754cb2e2403b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b0d3c700 1 -- 172.21.15.83:0/26281776 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a800b8a0 con 0x7f89ac15c820 2023-11-28T11:13:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b0d3c700 1 -- 172.21.15.83:0/26281776 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f89a800ba00 con 0x7f89ac15c820 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b0d3c700 1 -- 172.21.15.83:0/26281776 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a800bdb0 con 0x7f89ac15c820 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 -- 172.21.15.83:0/26281776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 msgr2=0x7f89ac0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac0c9b30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f89a80098f0 tx=0x7f89a8009c00 comp rx=0 tx=0).stop 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 -- 172.21.15.83:0/26281776 shutdown_connections 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac0c9b30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/26281776 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac06ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 -- 172.21.15.83:0/26281776 >> 172.21.15.83:0/26281776 conn(0x7f89ac0cf4e0 msgr2=0x7f89ac0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 -- 172.21.15.83:0/26281776 shutdown_connections 2023-11-28T11:13:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.502+0000 7f89b3fa2700 1 -- 172.21.15.83:0/26281776 wait complete. 2023-11-28T11:13:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.503+0000 7f89b3fa2700 1 Processor -- start 2023-11-28T11:13:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 -- start start 2023-11-28T11:13:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89ac06b290 0x7f89ac15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0ad280 con 0x7f89ac15c820 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac150f60 con 0x7f89ac06b290 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b3fa2700 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0ac670 con 0x7f89ac160fe0 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b153d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b153d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53980/0 (socket says 172.21.15.83:53980) 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.504+0000 7f89b153d700 1 -- 172.21.15.83:0/47007972 learned_addr learned my addr 172.21.15.83:0/47007972 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-28T11:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b153d700 1 -- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 msgr2=0x7f89ac0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b153d700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b1d3e700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89ac06b290 0x7f89ac15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b153d700 1 -- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89ac06b290 msgr2=0x7f89ac15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b153d700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89ac06b290 0x7f89ac15cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b153d700 1 -- 172.21.15.83:0/47007972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a8009550 con 0x7f89ac15c820 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b153d700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac150a20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f899c00e810 tx=0x7f899c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00c020 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f899c004250 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac15cbf0 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c0036a0 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.505+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac06a550 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.506+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac0ca070 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.506+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f899c022070 con 0x7f89ac15c820 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.507+0000 7f89a27fc700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89900682f0 0x7f899006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.507+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f899c085910 con 0x7f89ac15c820 2023-11-28T11:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.507+0000 7f89b1d3e700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89900682f0 0x7f899006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.507+0000 7f89b1d3e700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89900682f0 0x7f899006a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009c40 tx=0x7f89a8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.511+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f899c0522a0 con 0x7f89ac15c820 2023-11-28T11:13:09.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.738+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89ac074320 con 0x7f89ac15c820 2023-11-28T11:13:09.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.739+0000 7f89a27fc700 1 -- 172.21.15.83:0/47007972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f899c00f040 con 0x7f89ac15c820 2023-11-28T11:13:09.741 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-28T11:13:09.741 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/1656959851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1656959851},{"type":"v1","addr":"172.21.15.90:6833","nonce":1656959851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5740,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/3483483270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3483483270},{"type":"v1","addr":"172.21.15.90:6835","nonce":3483483270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/4103582904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4103582904},{"type":"v1","addr":"172.21.15.83:6836","nonce":4103582904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1603125533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1603125533},{"type":"v1","addr":"172.21.15.83:6837","nonce":1603125533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2023-11-28T11:13:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89900682f0 msgr2=0x7f899006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89900682f0 0x7f899006a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009c40 tx=0x7f89a8008030 comp rx=0 tx=0).stop 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 msgr2=0x7f89ac150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac150a20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f899c00e810 tx=0x7f899c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 shutdown_connections 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89ac160fe0 0x7f89ac0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89ac06b290 0x7f89ac15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:6832/103263,v1:172.21.15.83:6833/103263] conn(0x7f89900682f0 0x7f899006a7a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 --2- 172.21.15.83:0/47007972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ac15c820 0x7f89ac150a20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 >> 172.21.15.83:0/47007972 conn(0x7f89ac0cf4e0 msgr2=0x7f89ac160380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 shutdown_connections 2023-11-28T11:13:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-28T11:13:09.741+0000 7f89b3fa2700 1 -- 172.21.15.83:0/47007972 wait complete. 2023-11-28T11:13:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2023-11-28T11:13:09.754 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T11:13:09.754 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T11:13:09.755 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:13:09.755 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:09.763 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:13:09.763 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:13:09.763 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:09.812 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:13:09.813 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:13:09.813 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:09.825 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:13:09.825 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:13:09.825 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:09.876 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:13:09.877 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T11:13:09.877 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T11:13:09.877 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:09.915 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T11:13:09.916 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T11:13:09.916 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:09.987 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T11:13:09.987 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T11:13:09.987 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.058 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T11:13:10.058 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T11:13:10.058 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-28T11:13:10.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-28T11:13:10.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-28T11:13:10.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-28T11:13:10.125 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T11:13:10.126 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T11:13:10.126 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.166 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T11:13:10.166 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T11:13:10.166 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.247 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T11:13:10.248 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T11:13:10.248 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.322 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T11:13:10.323 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T11:13:10.323 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:10.342 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:10.388 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T11:13:10.389 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T11:13:10.389 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T11:13:10.389 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.413 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T11:13:10.413 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T11:13:10.413 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.438 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T11:13:10.438 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T11:13:10.438 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T11:13:10.438 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.465 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T11:13:10.465 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T11:13:10.465 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.489 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T11:13:10.489 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T11:13:10.489 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:10.540 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T11:13:10.541 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T11:13:10.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T11:13:10.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:13:10.573 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T11:13:10.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:13:10.875 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T11:13:10.960 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:13:10.961 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T11:13:11.048 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:13:11.048 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T11:13:11.121 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:13:11.121 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T11:13:11.368 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:13:11.368 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T11:13:11.466 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:13:11.466 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T11:13:11.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:13:11.541 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T11:13:11.614 INFO:tasks.ceph:Archiving mon data... 2023-11-28T11:13:11.615 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/data/mon.a.tgz 2023-11-28T11:13:11.615 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2023-11-28T11:13:11.634 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.0IzGirKttk 2023-11-28T11:13:11.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.0IzGirKttk 2023-11-28T11:13:11.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.0IzGirKttk 2023-11-28T11:13:11.873 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.0IzGirKttk is 397KB 2023-11-28T11:13:11.970 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.0IzGirKttk 2023-11-28T11:13:11.986 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/data/mon.b.tgz 2023-11-28T11:13:11.986 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2023-11-28T11:13:12.005 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.PrKxTA4Slu 2023-11-28T11:13:12.005 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PrKxTA4Slu 2023-11-28T11:13:12.200 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.PrKxTA4Slu 2023-11-28T11:13:12.287 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.PrKxTA4Slu is 517KB 2023-11-28T11:13:12.365 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.PrKxTA4Slu 2023-11-28T11:13:12.382 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/data/mon.c.tgz 2023-11-28T11:13:12.382 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2023-11-28T11:13:12.438 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.QymrrwWD4n 2023-11-28T11:13:12.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.QymrrwWD4n 2023-11-28T11:13:12.602 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.QymrrwWD4n 2023-11-28T11:13:12.689 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.QymrrwWD4n is 517KB 2023-11-28T11:13:12.789 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.QymrrwWD4n 2023-11-28T11:13:12.805 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T11:13:12.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:13:12.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:13:12.871 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T11:13:12.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:13:12.879 DEBUG:teuthology.orchestra.run.smithi090:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:13:12.926 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:13:12.941 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:13:12.947 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T11:13:12.949 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/remote/smithi083/crash 2023-11-28T11:13:12.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:13:12.978 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/remote/smithi090/crash 2023-11-28T11:13:12.979 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:13:13.013 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T11:13:13.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:13.024 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:13.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:13.131 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:13.157 INFO:tasks.ceph:Compressing logs... 2023-11-28T11:13:13.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:13:13.175 DEBUG:teuthology.orchestra.run.smithi090:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:13:15.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-28T11:13:15.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-28T11:13:15.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-28T11:13:15.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-28T11:13:17.643 INFO:tasks.ceph:Archiving logs... 2023-11-28T11:13:17.643 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/remote/smithi083/log 2023-11-28T11:13:17.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:13:18.423 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/remote/smithi090/log 2023-11-28T11:13:18.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:13:18.908 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T11:13:18.919 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T11:13:18.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:13:18.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:13:19.011 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T11:13:19.023 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T11:13:19.024 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:13:19.026 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:13:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:bash: ntpq: command not found 2023-11-28T11:13:19.043 INFO:teuthology.orchestra.run.smithi090.stderr:bash: ntpq: command not found 2023-11-28T11:13:19.045 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:13:19.046 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2023-11-28T11:13:19.046 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +816us[ +812us] +/- 80ms 2023-11-28T11:13:19.046 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -672us[ -675us] +/- 54ms 2023-11-28T11:13:19.046 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +180us[ +180us] +/- 73ms 2023-11-28T11:13:19.046 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:19.048 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:13:19.048 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2023-11-28T11:13:19.048 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 +836us[ +851us] +/- 80ms 2023-11-28T11:13:19.048 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -660us[ -646us] +/- 54ms 2023-11-28T11:13:19.049 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +204us[ +218us] +/- 73ms 2023-11-28T11:13:19.049 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:19.050 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T11:13:19.060 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T11:13:19.061 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T11:13:19.072 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:13:19.197 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:13:19.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:13:19.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:19.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:13:19.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:19.369 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T11:13:19.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T11:13:19.389 INFO:teuthology.task.internal:Duration was 1058.936903 seconds 2023-11-28T11:13:19.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T11:13:19.400 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T11:13:19.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:13:19.404 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:13:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:13:19.456 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:13:19.668 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T11:13:19.669 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2023-11-28T11:13:19.669 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:13:19.731 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2023-11-28T11:13:19.731 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:13:19.753 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T11:13:19.754 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:13:19.773 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:13:19.833 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T11:13:19.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:13:19.857 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:13:19.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T11:13:19.961 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T11:13:19.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:13:19.990 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:13:20.017 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T11:13:20.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:13:20.032 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:13:20.056 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2023-11-28T11:13:20.083 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2023-11-28T11:13:20.097 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:13:20.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:20.126 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:13:20.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:20.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T11:13:20.167 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T11:13:20.168 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/remote/smithi083 2023-11-28T11:13:20.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:13:20.206 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953/remote/smithi090 2023-11-28T11:13:20.207 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:13:20.244 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T11:13:20.244 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:13:20.246 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:13:20.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T11:13:20.309 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T11:13:20.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T11:13:20.322 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T11:13:20.323 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:13:20.325 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:13:20.339 INFO:teuthology.orchestra.run.smithi083.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:13 /home/ubuntu/cephtest 2023-11-28T11:13:20.351 INFO:teuthology.orchestra.run.smithi090.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:13 /home/ubuntu/cephtest 2023-11-28T11:13:20.353 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T11:13:20.366 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T11:13:20.396 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T11:13:20.421 DEBUG:teuthology.nuke:shortname: smithi083 2023-11-28T11:13:20.421 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:13:20.444 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:55.428636', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:13:20.447 DEBUG:teuthology.nuke:shortname: smithi090 2023-11-28T11:13:20.448 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:13:20.474 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469953', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:55.427071', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:13:20.501 INFO:teuthology.orchestra.console.smithi083:Power off 2023-11-28T11:13:20.501 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:13:20.528 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2023-11-28T11:13:20.528 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:13:20.550 INFO:teuthology.orchestra.console.smithi090:Power off 2023-11-28T11:13:20.551 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:13:20.566 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-11-28T11:13:20.585 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2023-11-28T11:13:20.585 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:13:20.613 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2023-11-28T11:13:24.567 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:13:24.614 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:13:24.678 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-11-28T11:13:24.725 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2023-11-28T11:13:28.682 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:13:28.725 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:13:28.792 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2023-11-28T11:13:28.836 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2023-11-28T11:13:28.894 INFO:teuthology.orchestra.console.smithi083:Power off completed 2023-11-28T11:13:28.937 INFO:teuthology.orchestra.console.smithi090:Power off completed 2023-11-28T11:13:29.041 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1058.9369032382965 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=363b88a520f44698996166f1fa938c05 status: fail success: false 2023-11-28T11:13:29.042 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:13:29.113 INFO:teuthology.run:FAIL